[iamfatness]

Closes the scripting half of #443: the release-gate outputs drill as one maintained command, scripts/qa/outputs-drill.ps1, replacing the one-off harness in D:\CoreVideo-QA\beta-20261021-build\artifacts\qualification-1008-25 (which passed the 2026-10-22 30-minute soak). Judging is not redesigned: it reuses the existing scripts/qa judges.

What it does

• Refuses if the app is running or port 8011 answers. Backs up production-output-preferences.json, points recording at <Out>\recordings, selects the ISO sources, forces RTMP off (the stored key and URL are never changed), and points SRT (caller) at a loopback ffmpeg listener on a free UDP port that the script starts itself. The file is restored byte for byte at the end, including on failure.
• Launches StartCoreVideo.cmd at Normal priority with COREVIDEO_QA_ISO_COUNTER / COREVIDEO_QA_PROGRAM_COUNTER and the delivery trace inside the evidence folder. It drives everything through /snapshot, /state and POST /invoke. No SendKeys, no elevation, no PresentMon.
• Readiness works like Wait-Ready (30 s stable, 300 s cap). Then it starts the virtual camera, the independent OS receiver and the check that the camera serves the installed DLL, and measures for -Seconds. Invariant breaks abort the run as FAIL. It stops in Collect-Identity order, stops the sink, leaves the meeting, closes the app with WM_CLOSE (the harness's exit path), restores preferences, and judges offline.
• verdict.json and summary.txt have one line per output (Program recording, each ISO, OS camera, stream): expected, decoded, missing and repeated identities, plus PASS, FAIL or NOT JUDGED with a reason. Overall PASS requires every enabled output to pass with zero missing; NOT JUDGED never counts as PASS. Program GPU deadlines are reported as supplementary evidence only.
• Without -RemoteIds, the drill uses local sources only: webcam and screen in Program, and the webcam as the only ISO. The verdict says so. Screen capture has no QA ISO identity stamp, so it is not ISO-recorded. Media is not included yet.
• iso-pixel-verdict.py gains additive missing/repeated counts. Its pass rule is unchanged.
• Docs: docs/qa/outputs-drill.md. CLAUDE.md is unchanged because its table covers docs/reference only. docs/BACKLOG.md is unchanged.

Exercised by tests (run locally, Windows PowerShell 5.1 + Python 3.12)

• scripts/qa/outputs-drill.test.ps1: 24/24 pass. It covers parameter validation (seconds range, D: evidence folder, remote ids, meeting URL, webcam id), id-list parsing, the workload plan in local and remote modes, webcam discovery from the profile, and the preferences backup/patch/restore round trip on a temp file with fake secrets. The round trip checks that RTMP is forced off, the RTMP key, URL and SRT passphrase are untouched, the ISO list stays a JSON array, non-ASCII text survives, older profiles get the missing keys, the restore is byte-identical and a corrupted backup is detected. It also covers the loopback sink URL, the free UDP port, readiness and collection invariants on sample snapshots, and the compact sample row, then runs the Python tests.
• scripts/qa/test_outputs_drill.py: 15/15 pass. It covers verdict aggregation from sample judge outputs shaped like the 2026-10-22 soak (PASS, FAIL on missing or repeated, judge error = NOT JUDGED, FAIL outranks NOT JUDGED, a collection failure is FAIL, unproven camera serving, camera interval coverage, summary lines). It also runs the offline orchestration end to end on synthetic 60-frame 1080p60 videos with QA identity bands and a synthetic trace, decoded by the real recording and ISO judges through ffmpeg: a clean run passes on Program, ISO and stream, and one dropped frame fails all three with missing=1.
• scripts/qa/test_iso_pixels.py (5/5, one new case) and scripts/tests/iso-pixel-evidence.test.mjs pass. The script's parameter refusals were run directly; they exit before touching anything.

Not verified

• The full command was not executed end to end. The live part (app launch, API driving, Zoom join, the loopback SRT sink, the OS camera receiver and the vcam-serve.log serving proof, stop order, app close, preferences restore against the real file) is a transcription of the reference harness and has not run from this script. I didn't run it because the owner may be using the machine and the test meeting has no remote feeds.
• The OS camera receiver is a dev build tool that the installer does not ship. Pass -CameraReceiver or build corevideo-shared-camera-receiver.
• Only tested on Windows PowerShell 5.1; pwsh 7 is not installed here.

🤖 Generated with Claude Code

[iamfatness]

Review fixes pushed in 19e47b1.

Blocker 1 (patch silently discarded). StreamSrtPort is now written as the string "50999". Every patched value is typed to the app model, and a patch that changes the JSON kind of any existing value is refused. After the API is up, the file is re-read (preferences-applied.json) and no output starts unless the patch survived. As runtime proof, readiness also fails hard if the recording writes outside <Out>\recordings or any RTMP sender appears. The .bak sidecar is backed up and restored too, including removing a sidecar the drill created.

I checked this against the installed app's own ProductionOutputPreferencesSerializer (beta-2026-10-22-q10191cd6d1e, loaded by reflection, read-only). A patched copy of the real v14 profile loads with folder, RTMP=false, port "50999" and host 127.0.0.1. The numeric-port variant comes back null, which is the reported Corrupt -> .bak path. That probe (scripts/qa/outputs-drill-prefs-probe) is now an optional case in outputs-drill.test.ps1.

Blocker 2 (WM_CLOSE while live). The stop sequence now lives in finally, which also runs on Ctrl-C:

• It stops the stream, then the recording, and waits up to 90 s for the recording to finalize. Then it stops the trace, the virtual camera and the engine, waits for the trace to finalize, and leaves the meeting.
• It sends WM_CLOSE only when the close-guard predicate is empty: not recording, not streaming, engine off, recording lifecycle settled, no live sender.
• Preferences are restored only after the app exits, waiting up to 10 min. Otherwise it writes preferences-restore-pending.json, prints a red warning and exits 3.

Medium 3. The reference's throughout-assertions are back, accumulated per sample into measurement.json:

• hardware Program and ISO encoders
• remotes at 1920x1080
• capture frames advancing within 2 s at 1920x1080 or larger
• Program source admissions
• stream: one sender, fps >= 55, no retry, no warnings

Any false flag fails the drill. Sample rows keep programFrame and programSourceAdmission, and the judge runs program-source-admission-capture.mjs and gates Program on it. Readiness has the sender fps and warnings checks again.

Medium 4. The drill runs at Normal priority for the whole live phase. The sink and the receiver are checked and their priorityClass is recorded.

Medium 5. A new status, UNBOUNDED: clean over the whole file but not camera-bounded. It never counts as PASS, so a -SkipVirtualCamera run can't PASS.

Lows 6, 7 and 9.

• Backups now live under %LOCALAPPDATA%\CoreVideoPro\outputs-drill-backups, not in the evidence folder.
• A dpapi: SRT passphrase is refused.
• The header wording about the RTMP key is corrected.
• The sink starts right after the port is chosen.
• The drill also refuses if corevideo-native or corevideo-zoom-engine is running.

Tests: outputs-drill.test.ps1 passes 37/37, including the installed-app serializer case. test_outputs_drill.py passes 23/23; its new end-to-end cases cover a camera-bounded PASS, a camera gap, UNBOUNDED when the camera is skipped, false or missing throughout flags, and an unadmitted Program source. test_iso_pixels.py passes 5/5. The real preference files were not touched (mtimes unchanged).

Still unverified: the drill has still not been run end to end against the installed app. That leaves the applied-check against a running app, the stop/close sequence and its close-guard predicate against the live /state, Ctrl-C behaviour, and the restore of the real files untested live.

🤖 Generated with Claude Code