Skip to content

test(v5): audit immutable development manuscript evidence - #325

Draft
TheHalfMoon wants to merge 13 commits into
docs/v5-development-manuscript-evidence-crosswalkfrom
quality/v5-manuscript-source-audit
Draft

TheHalfMoon wants to merge 13 commits into
docs/v5-development-manuscript-evidence-crosswalkfrom
quality/v5-manuscript-source-audit

Conversation

@TheHalfMoon

@TheHalfMoon TheHalfMoon commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Objective: fail-closed manuscript development evidence identity gate

This stacked DRAFT PR adds a read-only, deterministic, model-free audit of the already completed V5 development evidence and the previous development/manuscript crosswalk. It targets draft PR #324 (exact parent 6d2f1dd2e96ac1e1dc62a2743d7c11e8c6bce2a5). Scientific/claim/original frozen files, the working manuscript, original research branch and main remain unchanged under TheHalfMoon/commandMed.

Exact source/evidence safeguards

  • SHA-256 checks eight versioned committed evidence/text sources plus both original private Kaggle R1/R2 original ZIP bytes. Avoids relying on text-only provenance assertions.
  • Confirms R1/R2 independently verified immutable receipts: distinct named kernels version 1, 16,384 ordered rows and 8,192 source tasks in each, frozen exact public RiskCalcs source hash, no weights loaded during offline verification.
  • Checks repeatability95=0 only as measured for the four predeclared primary and three secondary metrics over two full same-configuration runs; preserves near-chance baseline and no cross-seed/hardware/clinical inference.
  • Confirms all four nonclinical benchmark-scale margins equal their pre-R1-output frozen floors, not selected after results.
  • Validates the entire descriptive B1/C1/C2 9-seed/intervention x 2-split x 4-primary-metric SD inventory (72 finite nonnegative source-task SD cells) and rejects false frozen primary family size/confirmatory power claims.
  • Re-hashes all 307 original frozen-entry files and fails closed on altered original manuscript, red-team, claim-ledger, frozen evidence or run inputs.
  • Output explicitly marks independent_peer_review=false, power_adequacy=NOT_QUALIFIED, confirmatory=false, clinical_validity=false, publication_authority=false.
  • Tests reject forged/wrong repeat identity, shifted margin, cherry-picked family, nonfinite SD, missing/mutated input or unwarranted confirmatory/clinical/publication status.

macOS exact-head qualification

  • New focused audit tests: 5 passed.
  • V5 tests: 310 passed + 86 subtests.
  • Full repository: 1,362 passed + 264 subtests.
  • Python compilation, git diff --check, execution of read-only audit against authentic files: PASS, including 307 hashes, 8 bound source files, original 2 ZIPs, 72 inventory cells.
  • Genuine Alibaba OCR zero-cost delegate eligibility preview: 1 implementation Python file eligible; tests and Markdown were excluded by default_path/unsupported_ext. This is NOT an external LLM review verdict.
  • Commit has DCO sign-off, but cryptographic signing requires separate verification. No false GitHub signed-commit assertion or user-account key change.

Hard boundaries

No new model runs, Kaggle reruns, private/PHI data, paid API, source/test sampling manipulation, new scientific hypothesis, calibration settings, domain-floor selection, confirmatory/reserve identity, cross-backbone replication, or publication. A passing model-free audit is not a substitute for independent scientific review. The exact primary non-target Holm family, seed-aware conservative variance methodology and 90% power gate are NOT FROZEN/QUALIFIED, and the claims remain unproven. Do not merge this or ancestor draft PRs without their genuine review, signature and founder gates.


Summary by cubic

Adds fail-closed, model-free audits of the V5 manuscript's locked development evidence, now run automatically on pull requests.

scripts/v5_manuscript_development_evidence_audit.py verifies SHA-256 hashes of nine versioned evidence sources plus both original R1/R2 Kaggle ZIP archives, re-checks all 307 frozen entry files, and validates the 72-cell descriptive SD inventory, the 72-cell development-only calculator ICC inventory, and repeatability metrics. scripts/v5_s1_source_case_pairing_audit.py replays the frozen rule-oracle source generator against the pinned public RiskCalcs JSON to confirm each source case maps one-to-one to a task ID (8,192 cases, 16,384 rows, two ordered prompt variants each) across both full R1 and R2 matrices. scripts/v5_s1_statistical_review_packet.py independently reconstructs the 72 paired-SD and 72 calculator-ICC cells algebraically and bridges them into 24 three-seed reviewer groups, retaining all 12 negative ICC estimates without qualifying power. The 307-entry manifest stays pinned under its own SHA-256, since it is not one of its own entries and could otherwise be rewritten alongside its references.

  • Adds a CI workflow that verifies exact-head checkouts, compiles all verifiers, retrieves the SHA-pinned public RiskCalcs blob, replays the original calculator-ICC inventory from the pinned source matrices, and runs all audits and their regression tests on pull requests touching V5 development evidence. The workflow pins numpy==2.4.6 to match the verified rule-oracle calculator and uses an immutable actions/checkout revision.
  • Adds three docs/ audit reports explaining scope and limitations, plus a generated reviewer-packet artifact.
  • Adds regression tests covering real evidence, scope escalation, forged margins, missing sources, invalid SD values, manifest pinning, ICC provenance and authority flags, negative ICC retention, and source-case pseudoreplication.

Both audits reject forged repeat identity, shifted margins, post-hoc power family claims, nonfinite SDs, missing or mutated inputs, and unwarranted confirmatory, clinical, or publication status. They mark power_adequacy=NOT_QUALIFIED and are not a substitute for peer review.

Written for commit 534120e. Summary will update on new commits.

View guided diff Turn on auto-fix

Signed-off-by: Gocceri <azialshehri@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Gocceri <azialshehri@gmail.com>
Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

Exact-head source-integrity hardening and public deterministic CI — 2026-10-09

Forward-only successor commits (normal push, no history rewrite):

  • 076f74cfa1e4ae7c0915a667a11af2b1a96a90d2: fixed an actual manifest-integrity gap. The 307-entry frozen source manifest was not itself one of those 307 entries; a coordinated change to the manifest and source files could have bypassed the old self-consistency loop. It is now independently SHA-256 pinned at c33e603e531b5f3957a0fbe5fa63d45f070e21e2243b268af232516990dd428f, with duplicate-entry rejection and a direct negative fixture.
  • eebeab9a89aa256534b5ef78e6d66fb820a0ab46: added a scoped, read-only, no-GPU, no-model GitHub Actions source-bound provenance gate. Uses checkout of the exact PR SHA, minimal contents:read permission, no persisted checkout credentials, Python compilation, full saved R1/R2/archive/307-file hash gate and regression tests. No PHI, weight access, Kaggle call, confirmatory/reserve generation or paid API.

Independent execution of deterministic checks on GitHub (not independent scientific/clinical review): GitHub Actions run #37980954005 completed success at exact head eebeab9, with every step green. Previous Apple Silicon exact-head local tests after fix were 1,363 repository tests + 264 subtests and 311 V5 tests + 86 subtests, plus six focused negative/positive audit tests. Source bindings still 307/307 PASS.

Unmet review/governance: This is not an external Alibaba OCR model verdict or independent methodologist review. GitHub previously classified the branch's DCO-only commits as unsigned; cryptographic signing and separate merge gates remain unresolved. Holm non-target family enumeration, seed-aware variance qualification and 90% power adequacy remain pending. PR is deliberately DRAFT / UNMERGED. No clinical, confirmatory, independent-family, release or publication claim is created.

Signed-off-by: Gocceri <azialshehri@gmail.com>
Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

V5 S1 original source-case → task pairing proven against full R1 and R2 — 2026-10-10

Added a genuinely model-free, full source-replay audit in this draft PR #325 (latest exact pushed HEAD 2e12096a3f21bdee25b41c356fa9a93a87f21d28) to address a research validity question: the frozen statistical unit is the source medical case, whereas the existing descriptive source-task SD inventory indexes task_id. We must not treat variants as independent clusters or assume a task ID equals a distinct case without proof.

Rebuilt from the pinned original source, not inferred from existing outcome labels: public ncbi-nlp/Clinical-Tool-Learning commit d474a95128e128623933c9be0d389ff7d82ef782, public RiskCalcs blob SHA-256 00a7a0089afffb66f2f32903bad94a5a2ea842841defb2d78e0686b0a5eb9ab9; pre-frozen 64-rule manifest SHA 445e37ee7b3a2bb56c21d04a6236e16a929f6207c74351c0143fe436c61abd9c; original generator code SHA-bound.

  • 64 selected synthetic rule calculators; 8,192 source-case IDs and 8,192 task IDs, each unique; 8,192 distinct calculator/state pairs. Exact split counts: TRAIN 256, DEV_EVAL 3,840, CAL_TUNE 256, CAL_EVAL 3,840. Each calculator 4/60/4/60. Frozen ordered task sequence SHA-256 36c86610e2a762cb57ab61a8dff6a829e7196a638178dc612b5d1d3d5bdd5e0c.
  • Every original source case was regenerated with its reference answer, target label, split, calculator, and canonical/transformed exact prompt hashes; all 16,384 rows in R1 and independently all 16,384 rows in R2 passed exact per-row identity matching. R1 matrix SHA 600dbd236098b967558e96d5ddf7d4144903fad96b172ad2871ec02fb0f30a44; R2 SHA 37a78d4d067d00a486e2c63fc72662ecb2cee4d54c522ac8e05048e432eec902. Frozen-case-to-task mapping SHA-256 43ce4be40bb5c1749dbbe21f6d8788dfd5c33bf72cea78936ec47011e72cf700.
  • This establishes a one-to-one mapping between case_id and task_id for the original prospectively frozen synthetic S1 development/calibration allocation, eliminating a specific potential case-vs-task pseudoreplication objection to the 7,680 DEV+CAL source-case descriptive rows. Canonical/transformed prompt variants remain one cluster, never two.
  • 13 new synthetic regression tests cover prompt/source/target tampering, missing/duplicated variant, duplicate intrinsic case/task/calculator-state identities and wrong source hash. Mac V5 suite 324 passed +86 subtests, full repo 1,376 passed +264 subtests, frozen file binding 307/307 pass. No original frozen file changed.
  • CI workflow now attempts to download only the SHA-pinned public RiskCalcs blob (no credentials), installs pinned public-rule evaluator dependency NumPy and runs the full regenerated case mapping in a read-only public GitHub runner. Initial Linux run #38025578858 FAILED solely because NumPy was absent; a forward-only commit 2e12096 pinned numpy==2.4.6 before the audit. New exact-head CI #38025668195 is queued as of this comment; do not claim CI success until GitHub reports it.

Scientific limit: one-to-one synthetic case/task identity does not validate clinical correctness, real-patient representativeness, independence of stochastic training seeds, conservative across-seed variance, a final frozen Holm non-target family, 90% prospective power, or any confirmatory/replication result. Existing negative/near-chance developmental findings are unchanged. No GPU/model inference, paid API, PHI or confirmatory/reserve access used. All PRs remain DRAFT; latest commit reports DCO only, GitHub cryptographic verification unsigned; independent scientific/OCR and founder authorization gates remain separate.

Copy link
Copy Markdown
Owner Author

2026-10-10 — Stacked dependency drift: PR #324 crosswalk changed after this exact-head integrity audit

Status: REPRODUCED READ-ONLY SHA-256 CONSISTENCY FINDING / NO CODE MUTATION / NO MERGE.

This draft PR #325 currently remains at exact head 2e12096a3f21bdee25b41c356fa9a93a87f21d28. Its model-free evidence verifier scripts/v5_manuscript_development_evidence_audit.py pins the crosswalk's original SHA-256 as cc70f46824269712135678de7a182da45a459f10bfc1aa19e4d65e4c9b4665ae.

Read-only inspection and an independently self-tested UTF-8 SHA-256 implementation confirm:

Required before any eventual merge: forward-only, provenance-preserving reconciliation of the #325 verifier binding and source semantics with the exact revised #324 crosswalk; update targeted negative regressions and rerun source-bound audit, all 307 frozen-entry checks and exact-head CI on the resulting branch. Ensure no mutable/unsigned source is quietly treated as the old snapshot. Do not solve this by disabling provenance checks, accepting multiple arbitrary hashes, force-pushing, rebasing or modifying any frozen evidence. Respect the existing independent review and cryptographic-signature/DCO gates (especially the separately recorded unsigned/mismatched sign-off on #324 head).

No Mac execution, new model inference, GPU, paid API, confirmatory/reserve case, new PR or scientific/clinical claim was involved in this finding. This is a future stacked-integration blocker, not evidence that #325's existing head or earlier exact-head CI was fraudulent.

Synchronize the downstream read-only evidence gate with the exact
source-bound upstream development-only manuscript crosswalk.
Preserve all 307 frozen original scientific bindings and no-go authority.

Signed-off-by: Abdulaziz M. Shehri <alshehriofficial@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-10 — Forward-only stacked evidence-binding reconciliation VERIFIED by exact-head CI

This supersedes the mechanical SHA mismatch blocker reported above, not the independent scientific/signature gates.

A normal forward-only commit c8006769a0460cabd3a2c4552db53cb8c0751b78 was added to this existing DRAFT #325 branch, parent 2e12096a3f21bdee25b41c356fa9a93a87f21d28; no new PR, force push, rebase or history rewrite was used. The exact two changed paths are:

  1. docs/research/paper-rebuild-v2-2026-10-02/V5_MANUSCRIPT_DEVELOPMENT_RESULTS_CROSSWALK_2026-10-09.md, synchronized byte-for-byte with docs(v5): reconcile manuscript with development-only evidence #324's dated calculator ICC follow-up (Git blob cb83fd6ba5c9ba3e2fe8848117ce752943c940b0).
  2. scripts/v5_manuscript_development_evidence_audit.py, whose LOCKED_SOURCES['crosswalk'] has the one exact new SHA-256 7ff52c5b616543c7d85a44e6ecbf9b982a95b2a58531cc17a66e002367d04c41, not a fallback/loosened hash list.

Genuine exact-head GitHub Actions: run #38073338441, event source head c8006769a0460cabd3a2c4552db53cb8c0751b78, COMPLETED SUCCESS. All job steps succeeded, including exact-head checkout, compiler check, both original full R1/R2 archives, fixed SHA-256 evidence and 307 original frozen source bindings, pinned public RiskCalcs source and source-case pairing replay, and fail-closed regressions. This was GitHub-hosted Linux CI; no Mac local suite or independent Alibaba OCR/Jev/Pstack/Graft/e2e qualification is claimed for this session.

Still NO-GO: GitHub reports new commit verification.verified=false, reason unsigned. It has a matching author/committer-name sign-off footer but no cryptographic signature. The earlier #324 author/signoff identity mismatch also persists in history. Full stacked review, independently accepted source/seed/calculator statistical model, true 90% Holm family power, rights/governance and founder confirmation still require separate accountable qualification under #326. This CI is integrity-only, not clinical or power evidence.

#325 remains OPEN / DRAFT / UNMERGED; main and every originally frozen scientific entry stay unchanged.

Forward-integrate the source-bound parent evidence, reject missing or altered ICC bytes, verify all 72 descriptive cells and preserve negative estimates plus independent scientific no-go flags. The original 307 frozen files remain unchanged.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-10 — SOURCE-COMPLETE ICC gate and exact-head CI success

New exact DRAFT PR head: eae26384628bc06c935f28f8abbad0519c867573, normal two-parent merge commit (parents c8006769a0460cabd3a2c4552db53cb8c0751b78 and updated parent-draft #324 882311d46676cf052f2536d0dd8ac1a259f3d108). This is draft-stack branch integration, not GitHub merging a gated research PR into main, and no history rewrite, force-push or reset occurred.

Verified defect and repair: the preceding crosswalk text referred to original calculator ICC development evidence that the downstream verifier did not require to exist. Now LOCKED_SOURCES['icc_inventory'] pins the exact checked-in original SHA-256 37d64c0f6ac14d04cc6420a020e7bb7997fb333e50729f33798b2d403d292370; absence or content drift fails closed. A new bounded semantic guard also checks all 72 expected intervention/seed/split/metric ICC cells, the 64 calculators × 60 development source-cases convention, reconstruction linkage to the prior paired-SD source hash, the 12 retained negative estimates and aggregate range/median, and explicitly false power, generalization, Holm and confirmatory-authority flags. Negative regressions reject spoofed authority, lost source hash, removed negative values, invalid cell identities and mutated bytes. The original 307 bound scientific entries are untouched.

Real Mac checks at resulting source before commit: full pytest 1,389 passed + 264 subtests, dedicated ICC/provenance tests passed, original source evidence verification PASS_DEVELOPMENT_ONLY_EVIDENCE_ADMISSION with nine locked development sources + two independent repeat archives, frozen 307/307, Python compile/diff checks passed. Genuine Jev v1.13.0 was invoked as an auxiliary bounded judgment (reported probability 0.02 that these ICC cells alone certify final scientific power); this is not independent methodological review. Genuine Graft no-LLM wiring indexed 68 Python files / 686 nodes / 1,723 edges, with its ephemeral graph cache removed from the repo. Alibaba Open Code Review 1.12.13 provided review eligibility preview only, four reviewable code/config/artifact files; no actual independent provider-backed OCR model verdict. Pstack executable was not present. The CLI source verifier was exercised end-to-end, but no clinical/model e2e experiment was authorized.

Genuine exact-head GitHub CI: run #38074111231, SHA eae26384628bc06c935f28f8abbad0519c867573: COMPLETED SUCCESS. Exact-head checkout, source hashes, original R1/R2 exports, 307 frozen files, SHA-pinned public RiskCalcs replay, source-case pairing audit and fail-closed tests all succeeded.

Hard gates still OPEN: local ED25519 Git signature verified with git verify-commit, but GitHub REST returns verification.verified=false, reason unknown_key; older unsigned and DCO-mismatch commits remain. No independent signed code/security, independent qualified scientific review, complete primary Holm family or crossed calculator/source-case/three-seed nuisance with true per-effect 90% power has been approved. Neither clean CI nor descriptive ICC permits clinical claims, paper publication, confirmatory/reserve access, or merging the gated drafts. Preserve #326 and existing founder/rights governance.

Fail closed when the development ICC result cannot be reproduced from original pinned matrices. Keep immutable evidence, 307 frozen files, no scientific claim promotion, and draft merge restrictions.

Signed-off-by: Gocceri <azialshehri@gmail.com>
Use the exact action revision already bound by the V5 calculator ICC workflow while retaining read-only checkout and source-bound offline verification.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-10 — Exact-head original-source ICC replay and immutable Actions revision (DRAFT ONLY)

Live exact head: 679fa728aadda2ec5e322f547b9dfe75fbd4f33a (normal forward-only commits following eae26384628bc06c935f28f8abbad0519c867573). No rebase, force-push, model/GPU experiment, scientific freeze modification, confirmatory/reserve identities, source overwrite, or change to main.

This advances the existing source integrity workflow .github/workflows/v5-manuscript-development-evidence-integrity.yml in two precisely scoped commits:

  • 9e30bd5270aa2fedbf2b97628fa1a20571710449: run the original pinned development calculator ICC analyzer with exclusive-create fresh output and compare the entire replayed JSON recursively to the immutable committed reference, preserving type/structure and with cross-platform 1e-10 relative/absolute tolerance for floating-point fields. Assert 72 source-bound cells, 72 source-SD reconstructions, 12 retained negative ICC observations and unchanged final_90_percent_power_qualified=false / confirmatory_authorized=false. Path filters and regression tests cover the new script.
  • 679fa728aadda2ec5e322f547b9dfe75fbd4f33a: replace floating actions/checkout@v4 in this workflow with existing project-qualified immutable revision actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 (v4.2.2), preserving persist-credentials: false and minimal contents: read.

Actual Mac verification: the analyzer independently replayed all original development source-bound matrices, yielding exact committed SHA-256 37d64c0f6ac14d04cc6420a020e7bb7997fb333e50729f33798b2d403d292370 and 72/72 reproducible cells, with 12 negative finite-sample estimates. The exact YAML CI replay step itself passed locally; YAML parsed; 32 focused tests passed; 1,389 full tests and 264 subtests passed on the code head before the final workflow-only checkout pin, followed by 19 focused regression tests and YAML verification after that pin. Diff/whitespace scope checks passed. Original 307 frozen scientific bindings remained unchanged. No other session's uncommitted work was staged, reset or overwritten.

Real GitHub-hosted exact-head qualification: workflow run 38074952765 COMPLETED SUCCESS at 679fa728aadda2ec5e322f547b9dfe75fbd4f33a. Every action job step succeeded, including SHA-pinned upstream RiskCalcs retrieval, original R1/R2 archive+307-file audit, full source-case pairing, original ICC replay, and fail-closed tests. Earlier run 38074750848 also succeeded at 9e30bd5.

Gates unchanged: this is development-only integrity evidence; no statistically approved complete primary Holm family, no validated source/calculator/three-seed variance or 90% power at 4096 cases, no independent scientific/methodology approval, no genuine independent Alibaba OCR/Jev/Pstack/Graft/e2e review certificate and no publication/clinical authorization. The new commits are locally SSH-signed and DCO-signed off, but GitHub reports verification.verified=false, reason=unknown_key. Earlier unsigned or mismatched author/sign-off commits remain in history. GitHub account signing-key registration and all governance/review conditions remain separate gates. Existing DRAFT PR #325 stays OPEN / NO MERGE and issue #326 remains open.

Read-only git merge-tree checks showed no textual conflicts between the updated #325 commit and future #327 code, or between #327 and #328; this does not qualify the active, separate uncommitted worktrees or later stack for merging.

Reconcile 72 original paired-SD and calculator-ICC development cells into 24 complete three-seed reviewer groups, retain all 12 negative ICC values and explicit unreviewed Holm/crossed-variance/power authority. Include immutable source replay, 307-entry audit, fail-closed mutation tests, CI and review documentation. No model inference, confirmatory access or scientific approval.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-10 — source-bound 24-group/72-cell V5 independent-statistics reviewer packet (development only)

Exact head: 44d892010b9477b4acfb2446282507d29a34bf99. Normal fast-forward to existing DRAFT PR #325, no new PR, no rebase/force-push, no alteration of the 307 original bound research/evidence files, no model/GPU/API/PHI or confirmatory/reserve experiment, and no main/frozen-branch merge.

Added outputs (five changed/additive files):

  • scripts/v5_s1_statistical_review_packet.py reads immutable SHA-256 pinned developmental paired-SD and calculator ICC evidence, runs the full 307-file source audit, and reconciles exact original seed-effect means across independently stored artifacts. It fails closed on duplicate/missing seed/metric/split records, mismatched delta means, tampered provenance, falsified science authority and erased negative ICC estimates. --verify compares immutable bytes; --output uses exclusive-create and will never overwrite preexisting evidence.
  • artifacts/v5/development/s1-statistical-review-packet-2026-10-10/reviewer-packet.json deterministic byte SHA-256 86a275cb5cd7760f421538df34fea8b6dc452a19d095ddc4f4c6f0e69a87e348. Full nonselective 24 B1/C1/C2 × DEV/CAL × canonical-accuracy/NLL/Brier/semantic-JS descriptive groups, 72 original source-case-seed records, observed 12 negative unclipped calculator ICC estimates preserved, per-seed raw-orientation paired mean, per-seed source-case SD, calculator ICC, and descriptive SD of 3 seed-level effect means; separate conditional-on-calculators and broader generalization not assumed. Plan remains 4096 unmaterialized source cases, while actual original development splits each have 64×60=3840 cases.
  • tests/reliability_v5/test_statistical_review_packet.py source replay, mutation/authority/seed-alignment negative tests and no-overwrite CLI regressions.
  • docs/research/paper-rebuild-v2-2026-10-02/V5_S1_DESCRIPTIVE_STATISTICAL_REVIEW_PACKET_2026-10-10.md independent reviewer decision packet clarifying the 6 outstanding gates.
  • Existing exact-head manuscript integrity GitHub Action reruns the source packet and compares complete generated JSON bytes, in addition to original evidence tests and 307-file audit.

Actual verification on authorized macOS: original 307 frozen entries, 9 pinned source blobs, R1/R2 two independent original archives audit PASS; deterministic packet --verify and the actual new CI bash step replayed successfully. 30 focused tests PASS; full repository 1,400 tests PASS + 264 subtests PASS; Python syntax/YAML/diff checks PASS. The original ICC analyzer remained separate and unchanged. Alibaba Open Code Review v1.12.13 local preview identified 3 relevant files and did not perform independent LLM/provider review; Jev 1.13.0 performed an auxiliary nonapproval check, likewise not peer review.

Live exact-head GitHub Actions: both independent run #38080357632 (source-bound manuscript integrity plus packet replay) and run #38080357624 (original calculator ICC source replay) COMPLETED SUCCESS at 44d892010b9477b4acfb2446282507d29a34bf99.

Scientifically blocked, explicitly unchanged: caller cannot infer/declare primary non-target Holm family/size from these data; true crossed covariance among 64 selected calculators, source cases and 3 training seeds is unqualified; no 90%-power certificate at 4096 cases; no clinical, publication, confirmatory, reserve or independent review authority. Issue #326 remains OPEN. Latest Mac commit locally SSH signed+DCO but GitHub marks verification.reason=unknown_key, and previous stack commit history remains as-is. PR stays DRAFT and UNMERGED.

Cross-check all original ICC moments and paired source-case sample SDs algebraically, including the negative finite-sample ICC estimates. Add fail-closed corruption tests and documentation; preserve source-bound packet bytes, original 307 frozen sources and every unapproved statistical authority flag.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-10 — independent ANOVA identity checks added to the source-bound reviewer packet (DRAFT)

Exact new head: 534120efd04482c4bbd4f73d1add2fc0f624ed0b, forward-only successor to 44d892010b9477b4acfb2446282507d29a34bf99. Only three preexisting files were edited: reviewer generator, focused fail-closed tests, and methods documentation. The frozen 307 evidence entries, nine immutable sources, original ICC results and byte-identical committed reviewer packet SHA-256 86a275cb5cd7760f421538df34fea8b6dc452a19d095ddc4f4c6f0e69a87e348 are unchanged.

New mechanically falsifiable review protections: each of 72 stored source-case sample SD cells is recomputed independently from original calculator ANOVA mean-square components, sample_sd = sqrt(((K-1)*MS_B + K*(m-1)*MS_W) / (K*m-1)) with original K=64, m=60; each stored unclipped ICC is reconstructed as (MS_B - MS_W)/(MS_B+(m-1)*MS_W), without erasing any of the 12 negative finite-sample values. Nonfinite/negative ANOVA components, mismatched ICC, inconsistent SD even if both MS components are proportionally scaled (which would preserve rho), and forged status fail closed. Full original development evidence had max SD reconstruction absolute difference 3.552713678800501e-15, 72/72 matched, and ICC moment checks matched the committed values.

Actual Mac verification: 34 focused source packet/integrity/ICC tests passed, including new independent corruption fixtures. Full repository 1,404 tests + 264 subtests passed. Exact YAML CI replay step and byte-identical packet verification passed. Only authorized Mac was used, no model/GPU, external paid API, reserved/confirmatory data, source/branch rewrite, or changes to another worktree's dirty files.

Exact-head GitHub CI success: source-bound audit #38081017970 SUCCESS and original ICC replay #38081017999 SUCCESS, both at 534120efd04482c4bbd4f73d1add2fc0f624ed0b. Local ED25519 commit signature verified and matching DCO signoff exists, but GitHub still reports verification.verified=false, reason=unknown_key, as the Mac signing public key is not registered among this account's GitHub signing keys. No account key/security settings were modified.

Authority boundaries unchanged: this proves internal consistency/reproducibility of development evidence only, not a valid crossed source-case/calculator/training-seed covariance model, a reviewed complete primary Holm family or 90% confirmatory power at 4,096 prospective source cases. Real qualified independent science/code/provenance review and founder governance under #326 remain pending. The PR stays OPEN DRAFT, no merge.

Bind seven original preregistered intervention candidates to eight frozen draft assurance axes for exhaustive independent nomination review, explicitly without creating a 56-test Holm family or false power authority. Include 307-source audit, deterministic evidence, fail-closed tests and exact-head CI replay.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-11 — add source-bound undecided non-target hypothesis nomination coverage

Exact draft head: 6fdc33c77fa072e47efb056bf505a75dc005bd45. Added source-hash-locked read-only reviewer aid: candidate coverage JSON, interpretation guide, scripts/v5_holm_family_coverage_inventory.py, regression tests and exact-head CI replay. Output byte SHA-256 bbd1c687b7336ddc3fcf0f501c066591c8ab99b77b61f9be2e101ac2f19dde84.

Scope: enumerates all 7 named prospective draft intervention IDs from original preregistration-v5.md and all 8 property axes from original intervention-matrix-protocol.md as 56 possible review coverage slots, NOT 56 hypothesis tests. Explicitly marks every pair UNDETERMINED for inclusion, applicability, target/non-target hard-gate, native metric, population, policy, margin, coupling, etc. Unqualified descriptive training-seed availability is marked for B1/C1/C2 only, not as inferential approval; missing A1/A2/D1/D2 descriptions are not discarded or labeled passing. The actual primary Holm family size is null, as are reviewed primary members. Every authority flag remains false.

Real verification on authorized macOS: 307 frozen scientific files hash-verified without modification, original source text bytes and nine source-bound artifacts pinned, 8 new source-grid tests PASS; 31 focused tests PASS, 1,412 full-repository tests + 264 subtests PASS. Exact workflow bash replay compares generated payload bytes. Hosted exact-head GitHub Actions #38165661342 and #38165661351 both COMPLETED SUCCESS. No new model/confirmatory/reserve source IDs, paid compute, changing clinical claims, merge to main or false review.

Review decision still open: an independent qualified methodologist must explicitly disposition these possible pairs and specify any additional native-metric hypothesis cells, then prospectively approve full Holm membership, crossed 3-seed/calculator/source-case covariance/missingness, margins and meaningful 90% power. Locally valid Mac SSH signature remains GitHub unknown_key. PR stays DRAFT/UNMERGED; Issue #326 remains OPEN.

Use same-directory synced temporary files and no-clobber hardlink publication in reviewer packet and candidate coverage generators; cover short writes, disk failures, fsync failures, destination collisions and symlink preservation with synthetic regressions. Preserve source hashes, 307 frozen originals and all unapproved Holm, power and confirmatory flags.

Signed-off-by: Gocceri <azialshehri@gmail.com>

Copy link
Copy Markdown
Owner Author

2026-10-11 — no-clobber complete publication of source-bound development reviewer artifacts

Exact new head: d1a805f769536e7892b4ad2336f4907596ee6ce7, forward-only addition to existing DRAFT PR #325. The original --output writers for the 72-cell statistical reviewer packet and 56-slot undecided Holm candidate coverage both used exclusive-create file mode, but a disk-full or mid-write failure could leave a partially written new evidence filename; no overwrite guarantee is not the same as complete-file publication.

Repair: shared, model-free scripts/v5_development_evidence_writer.py stages bytes in a restrictive same-directory temporary file, handles short writes, synchronizes bytes with fsync, and publishes the completed destination via a no-clobber hard link; always removes the temp name. No unsafe rename/replace fallback. Unsupported hardlinks fail closed; no promise of parent-directory fsync or absolute crash durability. Both reviewer generators use the same audited helper. Synthetic tests reject an existing file or symlink, a zero-byte write, partial-write/disk-full, fsync failure, publication failure, and verify exact short-write retries and no residual temp files.

Actual authorized Mac evidence: 38 focused tests PASS, 1,419 full repo tests and 264 subtests PASS, both exact committed evidence artifacts replayed byte-for-byte (their SHA-256 bbd1c687b7336ddc3fcf0f501c066591c8ab99b77b61f9be2e101ac2f19dde84 and 86a275cb5cd7760f421538df34fea8b6dc452a19d095ddc4f4c6f0e69a87e348 unchanged), all 307 original scientific source files pinned, CI YAML/test inclusion and compilation checks PASS. Hosted exact-head GitHub ICC CI 38169348486 and manuscript CI 38169348511 both COMPLETED SUCCESS. No experiment/model/GPU, extra case IDs, source-scientific freeze edits, costly cloud, or main merge.

Gates: all complete non-target family, multi-seed crossed nuisance and 90% statistical power fields remain null/false, pending qualified external review. Local ED25519 signature is valid but GitHub key registration remains incomplete; no GitHub account or signing key settings changed. PR stays DRAFT/OPEN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants