Commit test-evidence projections instead of raw documents, and remove identity leaks from the vscode MSTest tooling

[drmoisan]

Suggested title

Commit test-evidence projections instead of raw documents, and remove identity leaks from the vscode MSTest tooling

Summary

• The two vscode MSTest entry points no longer leave raw evidence as the committed artifact. The coverage entry point now writes a package-level JaCoCo projection beside the raw Cobertura document and both entry points write a test-result summary derived from the raw test-result document.
• The raw coverage document is discarded under a stated invariant: it is kept when the resolved output directory is the repository coverage directory, and discarded when it is any other directory. The discard runs only after the threshold assertion, the projection write and the reconciliation assertion have completed.
• The projection is reconciled against its source rather than trusted. Assert-JacocoProjectionReconciliation throws unless the summed package LINE counters equal the source document's root covered-lines and valid-lines attributes.
• Both entry points now pass an explicit results directory and an explicit trx log file name, so the test console stops emitting a test-result document named from the operator account and host.
• Four configuration and documentation corrections remove identity leaks that live in configuration rather than in historical evidence, and CLAUDE.md gains a section stating the permitted committed test-evidence formats.
• All 23 acceptance criteria in the feature spec are satisfied and checked off.

Why

The maintainer recorded a committed test-evidence convention on #671 on 2026-09-11. Before this change the two entry points committed raw evidence: a raw test-result document carrying the operator account name, the host name and the absolute checkout path across six attributes, and a raw Cobertura document measured in tens of megabytes.

This item implements that convention and removes the identity leaks that sit in configuration. The tooling half of #602 and the configuration leaks tracked under #728 are in scope. The historical sweep of evidence already committed from earlier features is deliberately not in scope, and is discussed under Follow-ups.

CLAUDE.md is the home for the convention text because the evidence-and-timestamp conventions document and the atomic-plan contract are both push-down owned, so an edit to either would be reverted on the next push-down. The repository-root instruction file is owned here and is loaded into every agent session.

What Changed

Core logic

• scripts/vscode/Invoke-MSTestWithCoverage.Projection.ps1 — new. Holds ConvertTo-JacocoPackageProjection (pure writer), Assert-JacocoProjectionReconciliation (pure assertion) and Test-RawCoverageDocumentRetained (pure predicate). It is a new part file rather than an addition to the helpers file because the helpers file had 29 lines of headroom against the repository's 500-line ceiling.
• scripts/vscode/Invoke-MSTest.TrxSummary.ps1 — new. Holds Get-TrxRunSummary (pure reader over a test-result document string) and Format-TrxRunSummary (pure formatter). Dot-sourced by both entry points.
• scripts/vscode/Invoke-MSTestWithCoverage.ps1 — the argument builder gains the results-directory and trx-logger switches on the test-console segment; the main function writes the projection, runs the reconciliation assertion, applies the conditional discard and writes the test-result summary.
• scripts/vscode/Invoke-MSTest.ps1 — the same two switches on the plain path, plus summary write and raw-document discard.
• scripts/vscode/Invoke-MSTestWithCoverage.Helpers.ps1 — one added dot-source line.

Tests

Four new Pester files and three existing files repaired. The three repairs are forced by the signature changes rather than chosen: adding two mandatory parameters breaks parameter binding at every existing call site, and array assertions change because the returned arrays gain two elements. Invoke-MSTest.RunSettings.Tests.ps1 was already 496 lines against the 500-line ceiling, so its call sites were converted to splatting, which is net line-reducing.

Configuration and documentation

• TaskMaster/TaskMaster.csproj — the publish-destination element now carries the repository-relative value the other project file already uses for the same property.
• .vscode/settings.json — the Power Query additional-symbols array element now begins with the editor workspace-folder variable and contains no drive letter.
• CLAUDE.md — a new ## Committed Test Evidence Format section, and both test-console toolchain steps now name the explicit results-directory switch and log-file-name form.
• Six agent-memory documents — five token substitutions plus one deliberate rewrite, and two added rules in the shared hygiene document.

Architecture / How It Fits Together

The design keeps the new logic pure and testable and leaves the entry points as thin wiring.

The coverage entry point resolves its output path and results directory by joining the supplied parameters against the repository root, runs the collector, post-processes the raw Cobertura document, then calls the projection writer on the post-processed content. The reconciliation assertion runs against the same post-processed source, so a projection that disagrees with its source fails the run rather than being written silently. The summary reader parses the test-result document and the formatter renders it. Only then does the retention predicate decide whether the raw document survives.

The ordering matters and is asserted in-process by a call-order test, because an end-to-end run emits no ordered progress output from which an order could be read.

Verification

Completed

PowerShell gates, recorded in this feature folder's evidence tree:

• Format, analyzer and Pester gates pass. The analyzer comparison is set-based against the Phase 0 baseline rather than exit-code-based, because the analyzer exits 1 on any warning and a pre-existing plural-noun warning already exists in the helpers file.
• New-code line coverage for the two new part files is at or above 90 percent for each, derived per file from the emitted JaCoCo document rather than from the runner's aggregate percentage.
• Every PowerShell file in the change is at most 500 lines, and the helpers file grew by one line.

C# gates, re-run after this branch merged origin/main:

• dotnet tool run csharpier check . — exit 0, 1627 files checked.
• Analyzer rebuild — exit 0, 0 warnings, 0 errors.
• Nullable rebuild — exit 0, 0 warnings, 0 errors.

All three match their Phase 0 baselines exactly. The merge altered C# compilation inputs, so these were re-run rather than carried forward from the pre-merge measurement.

End-to-end observation, two full runs of the coverage entry point over every discovered Debug test assembly:

• Default output: 7222 tests, 7222 passed, 0 failed. Raw document retained, projection written beside it, reconciliation exact at 56066 covered and 65416 valid lines, test-result summary present.
• Output pointed at a directory other than the repository coverage directory: 7222 tests, 7222 passed, 0 failed. Raw document discarded, projection and summary retained.
• Neither run produced a test-result document bearing the default account-and-host name. Zero matches over both results-directory listings and zero over the changed-path union.

Repository coverage measured during the first run, against the CLAUDE.md floors of 80 percent line and 75 percent branch: first-party line coverage 85.71 percent and branch coverage 79.87 percent. Both clear.

Recommended

• pwsh -NoProfile -File scripts/vscode/Invoke-MSTestWithCoverage.ps1 to reproduce the retention path.
• The same entry point dot-sourced, calling Invoke-MSTestWithCoverageMain with -CoverageOutput and -ResultsDirectory pointed at a directory other than the repository coverage directory, to reproduce the discard path.

Backward Compatibility / Migration Notes

• No public API is removed. Both argument builders gain two parameters, which is a breaking change for any in-repo caller; all in-repo callers are updated in this change and the three affected test files are repaired here.
• The coverage entry point's ResultsDirectory parameter is exposed on the main function rather than on the script's top-level parameter block. A caller needing to override it dot-sources the script, which suppresses auto-invocation, and calls Invoke-MSTestWithCoverageMain directly.
• Behaviour change for operators: the raw coverage document is no longer guaranteed to survive a run whose output is directed outside the repository coverage directory. Retention inside that directory is preserved deliberately, because the repository ignore file's own comment records that the editor coverage extension reads it.

Risks and Mitigations

• Risk: the projection could silently disagree with the document it summarizes. Mitigation: the reconciliation assertion throws on disagreement and names both the expected and observed totals; both end-to-end runs reconciled exactly.
• Risk: the discard could run before the projection is written, destroying the only copy. Mitigation: an in-process call-order test pins the discard after the threshold assertion, the projection write and the reconciliation assertion.
• Risk: the three repaired test files could mask a regression through mechanical edits. Mitigation: no new describe block was added to any of them; new behaviour is asserted in the four new test files instead, so each existing file grows by the minimum the signature change forces.
• Rollback: the change is additive in two new part files plus wiring, so reverting the wiring in the two entry points restores the previous behaviour without touching the new files.

Review Guide

Suggested order:

1. scripts/vscode/Invoke-MSTestWithCoverage.Projection.ps1 and scripts/vscode/Invoke-MSTest.TrxSummary.ps1 — the new pure logic, and the place where correctness is decided.
2. The wiring in scripts/vscode/Invoke-MSTestWithCoverage.ps1 and scripts/vscode/Invoke-MSTest.ps1, particularly the ordering around the discard.
3. The four new test files.
4. The three repaired test files. Invoke-MSTest.RunSettings.Tests.ps1 carries a splatting conversion across many call sites and is the noisiest diff in the change; it is mechanical.
5. The configuration and documentation corrections.

Largely mechanical and safe to skim: the feature folder's evidence tree, which is the bulk of the changed-file count.

Note on the diff size: this branch merged origin/main, so the changed-path union against the original base anchor spans 195 paths, of which 91 are inherited from two items that merged to main after the anchor and are not authored here. The P6-T4 evidence artifact reconciles that figure against the earlier 100-path inventory in full.

Follow-ups

• The historical sweep of test-result documents already committed from earlier features is tracked separately. More than one hundred are tracked and at least one still carries an unredacted default name, so it is deliberately excluded here; a tree-wide scan would fail regardless of this change. That sweep should run after this merges so a fresh run does not reintroduce the prefix.
• Citing the new convention section from the atomic-plan contract's evidence tasks is an upstream change, because that contract file is push-down owned and an edit here would be reverted.
• No feature-review artifacts (code-review, feature-audit, policy-audit) were produced for this item.

GitHub Auto-close

• None

The context bundle reports GitHub validation unavailable and lists no verified autoclose issue, so no auto-close bullet is emitted. This work implements issue #873. The issue numbers #602, #646, #662, #671, #718 and #728 appear in the feature documents as background references and as scope boundaries; they are not auto-close targets for this pull request and should remain open.