test(server): remove replay harness self-tests - #13531
Merged
Merged
Conversation
Replay fixtures that drive the real adapters through the whole orchestrator are the coverage for the replay harness: if a runner, gate, controller, or transcript loader misbehaves, the fixture suite fails. Tests that assert the harness and recorders against hand-built transcripts only restate the harness. Delete the Claude, Cursor, and OpenCode testkit self-tests, the ProviderReplayGate and NDJSON loader tests, the Codex app-server replay transport test, the recorder script helper tests, and the fixture-input materializer and Codex canonical-dedupe helper cases. Keep the fixture registry contract checks. Unexport the testkit helpers only those tests imported. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
juliusmarminge
merged commit Sep 25, 2026
d41bac5
into
t3code/codex-turn-mapping
23 of 24 checks passed
Contributor
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — The PR removes replay-harness self-tests and narrows testkit exports while leaving helper implementations and production paths unchanged. Its impact is limited to test coverage and internal test infrastructure, with no product-default or static-analysis suppression changes. You can add or adjust custom eligibility rules. Learn more. |
juliusmarminge
added a commit
that referenced
this pull request
Sep 25, 2026
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The V2 test tree had tests for the replay harness itself: the Claude, Cursor, and OpenCode replay runners, the replay gate, the NDJSON loader, the Codex app-server replay transport, and the fixture recorders. The replay fixtures already run the real adapters through the whole orchestrator (
OrchestratorReplayFixtures.integration.test.ts), so a harness bug fails those. Testing the harness against hand-built transcripts only restates it.Removed
Named in the request
Adapters/ClaudeAdapterV2.testkit.test.ts(17 tests): replay gate hold and release, mismatch wakeups, interrupt bookkeeping, permission-callback abort and per-query scoping, cancelledruntime_exit, recorder interrupt and finalize behavior, denied-write allow substitution, andresolveClaudeRecordingExecutablePath.Adapters/CursorAdapterV2.testkit.test.ts(4 tests): runtime-instruction prompt mismatch, and wakeups when an outbound frame mismatches.Adapters/OpenCodeAdapterV2.testkit.test.ts(5 tests):OpenCodeReplayControllermessage-ID binding and abort handling.Other harness self-tests in the V2 tree
testkit/ProviderReplayGate.testkit.test.ts(2 tests). The gate holds frames in theclaude_background_*andqueued_cancelled_while_activefixtures.testkit/ReplayTranscriptNdjson.test.ts(7 tests). Every fixture loads through this decoder, with<workspace>placeholders in 37 transcripts.packages/effect-codex-app-server/src/replay.test.ts(2 tests). Every Codex fixture runs throughlayerReplay.apps/server/scripts/{acp-replay-agent,replayRecorderDeferredRegistry,cursorReplayRecordingWorkspace,codexReplayRecordingRecords,claudeReplayRecordingConfig}.test.ts(11 tests). These covered recorder and replay-agent helpers. A broken recorder shows up when you record or replay a fixture.OrchestratorReplayFixtures.contract.test.ts: dropped 6 cases that fed hand-built step lists to the fixture-input materializer (queue barriers, steer/answer ordering, run-ID derivation). The real fixtures cover each step type:queued_turn,queued_cancelled_while_active,plan_questions,message_steering, andclaude_background_subagent_after_root.CodexReplayFixtures.integration.test.ts: dropped the case that fed a made-up conflict to the test file's own dedupe helper.The testkit helpers that only these tests imported are no longer exported. Nothing else about them changed, and the recorders and harness still call them internally.
Kept, and why
OrchestratorReplayFixtures.contract.test.tskeeps its two fixture-registry checks: every fixture has a transcript, a matching scenario and provider, unique command IDs, decodable commands, and gate labels that exist in the transcript, and Codex transcripts start and end at the app-server boundary. These guard the fixture set itself.CodexReplayFixtures.integration.test.tsandClaudeReplayFixtures.integration.test.tskeep their checks that transcripts match the real protocol and that the fixture set is complete.scripts/acpMockCancellationState.test.tsstays. It tests the ACP mock agent that the adapter and provider unit tests use, not the replay harness.Moved
Nothing.
resolveClaudeRecordingExecutablePathis recorder-only. The production function it wraps,resolveClaudeSdkExecutablePath, keeps its tests inprovider/Drivers/ClaudeExecutable.test.ts, including shim-to-package-entry resolution.permissionResultFromDecisionkeeps its tests inClaudeAdapterV2.test.ts.Line delta: +24 / −2218 across 16 files.
Verification
vp test runforOrchestratorReplayFixtures.integration,OrchestratorReplayFixtures.contract,CodexReplayFixtures.integration,ClaudeReplayFixtures.integration,OrchestratorReplayRecovery.integration,ClaudeAdapterV2,CursorAdapterV2,OpenCodeAdapterV2, andClaudeExecutable: 9 files, 264 tests passed.vp exec tsc --noEmit -p .inapps/serverand inpackages/effect-codex-app-server: noerror TSorwarning TS.vp run knip:check: clean. The baseline before the change was also clean.vp linton the touched files: one warning, the unusedmakeClaudeAgentSdkReplayQueryRunnerLayerinClaudeAdapterV2.testkit.ts. The same warning appears at the base commit, so this change did not introduce it.Model: Claude Opus 5.5 (Claude Code)
🤖 Generated with Claude Code