Skip to content

test(server): record the Codex thread rollback fixture live on 0.156.1 - #13505

Merged
juliusmarminge merged 1 commit into
t3code/codex-turn-mappingfrom
v2/codex-rollback-live
Sep 24, 2026
Merged

juliusmarminge merged 1 commit into
t3code/codex-turn-mappingfrom
v2/codex-rollback-live

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

The Codex thread_rollback transcript was captured on Codex 0.120 and its rollback exchange was hand-edited into the 0.156 thread/read + thread/turns/list + thread/revert shape, so nothing proved the revert path against a real app-server. This re-records it live on Codex 0.156.1.

What changed

  • Fixture: thread_rollback/codex_transcript.ndjson is now a straight recording from Codex 0.156.1. Every frame is live and nothing was hand-edited.

  • Recorder (record-codex-app-server-replay-fixture.ts):

    • sends the adapter's initialize notification opt-out and turn/start defaults (never-approve, full-access sandbox, cwd, model, detailed summary, user reviewer);
    • reads the thread before reverting, the same way the adapter checks the history mode;
    • takes --model / T3_CODEX_REPLAY_MODEL;
    • writes the recording cwd as <workspace> in outbound frames (codexReplayRecordingRecords.ts).

    Together these make the transcript replay against the adapter without edits.

  • Model: the fixture's Codex variant gets its own modelSelection (gpt-5.6-luna), because the hub no longer serves gpt-5.4. The shared CODEX_MODEL_SELECTION is unchanged.

  • Assertions (codex_output.ts):

    • check that the answer after the rollback ("repeat the conversation") includes the first turn and not the reverted second one, which proves Codex's own context was rewound;
    • accept the two reasoning items the current model emits.

What the live recording showed

The recording agrees with the adapter's assumptions; nothing contradicts them:

  • thread/start returns historyMode: "paginated" with no historyMode param, so threads T3 starts are revertable.
  • thread/turns/list (descending, limit 1) returns the second turn, and thread/revert {beforeTurnId} succeeds. Its response carries thread (with turns: []), turnsBackwardsCursor and itemsBackwardsCursor.
  • During the revert, Codex unloads and reloads the thread. It emits thread/status/changed notLoaded → idle, then sends thread/reverted after the response.
  • The recording also has a model-mismatch warning during the reload. The recorder causes it, not Codex: it sends thread/start with {}, so the thread starts on the hub's default model, and the revert reloads it with a mismatch warning. The adapter sends model on thread/start, so users likely don't see it. A follow-up will make the recorder send model/cwd on thread/start.
  • The next turn's "repeat the conversation" answer only contains the first exchange.

Verification

  • vp test run src/orchestration-v2/testkit/OrchestratorReplayFixtures.integration.test.ts -t thread_rollback: 2 passed (Codex and Claude).
  • vp test run on CodexReplayFixtures, ThreadFork, ThreadMergeBack, OrchestratorReplayFixtures.contract, CodexAdapterV2.test.ts and scripts/codexReplayRecordingRecords.test.ts: 139 passed.
  • vp exec tsc --noEmit -p . in apps/server: no errors or warnings. vp lint and vp fmt on the touched files are clean.
  • vp run knip:check passes.
  • Recorded with T3_CODEX_BIN=<codex 0.156.1> node scripts/record-codex-app-server-replay-fixture.ts --scenario thread_rollback --model gpt-5.6-luna.
  • Not run: the other Codex fixtures, which still pin gpt-5.4 and pre-0.156 recordings.

Model: Claude Opus 5.5 (Claude Code)

🤖 Generated with Claude Code


Devin Review

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Sep 24, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 0cc5e27

Macroscope's review found this PR approvable — This PR refreshes a Codex rollback replay fixture and its development-only recording harness, adding assertions and workspace normalization without changing production request-path behavior. The large diff is primarily regenerated test transcript data.

No code changes detected at e37e762. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 4.9 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.4 KiB — 29.3 KiB ✅
Codex Live turn messages — 2 — 8 ✅
Claude Total thread wire — 4.9 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 20.8 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: e37e762 · Source CI: failure

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

The Codex thread_rollback transcript was captured on 0.120 and its rollback
exchange was hand-edited to the 0.156 thread/turns/list + thread/revert shape.
Re-record it against a real 0.156.1 app-server so the revert path is proven
by Codex itself.

The recorder now sends the adapter's initialize opt-out and turn/start
defaults, reads the thread's history mode before reverting (as the adapter
does), accepts --model, and writes the recording cwd as <workspace>, so the
transcript replays without edits. The fixture's Codex variant uses
gpt-5.6-luna because the hub no longer serves gpt-5.4.

The live recording shows a paginated thread from thread/start, a revert
response with empty turns plus backwards cursors, and a post-revert recall
that only contains the first turn. The output assertion now checks that
recall, and the extra reasoning items the current model emits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@juliusmarminge
juliusmarminge merged commit 2a169ae into t3code/codex-turn-mapping Sep 24, 2026
23 of 24 checks passed
@juliusmarminge
juliusmarminge deleted the v2/codex-rollback-live branch September 24, 2026 22:19
juliusmarminge added a commit that referenced this pull request Sep 24, 2026
#13505)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
juliusmarminge added a commit that referenced this pull request Sep 25, 2026
#13505)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant