Skip to content

0.5.1: re-verified against claude 2.1.267, codex 0.154, copilot 1.0.88, grok 1.0.40 - #38

Merged
pathscale merged 5 commits into
masterfrom
fix/verify-current-clis
Sep 23, 2026
Merged

pathscale merged 5 commits into
masterfrom
fix/verify-current-clis

Conversation

@pathscale

Copy link
Copy Markdown
Owner

Releases 0.5.1. The crate is re-verified against the installed CLIs: claude 2.1.267, codex-cli 0.154.0, GitHub Copilot CLI 1.0.88 and grok 1.0.40. Every flag was checked against each CLI's --help, and then the live suite was run once against all four.

Fixed

  • Codex: resuming with extra directories failed. codex exec resume refuses --add-dir (unexpected argument '--add-dir'). A resumed run now carries the directories as -c sandbox_workspace_write.writable_roots=[...]; a fresh exec still uses the flag. Has a unit test.
  • Codex: app-server errors lost their message. The notification is { error: { message }, willRetry }. The crate read a top-level message and failed the turn even when codex said it would retry. Has a unit test.
  • Claude: a [1m] run sometimes reported no window. This was caught by the live suite on 2.1.267. --output-format json has no init record, so when claude's Haiku helper ran there were two modelUsage entries and no name to choose between them. The crate now picks the entry whose counts equal the top-level usage, and only when exactly one does. Has a unit test.
  • Copilot: effort now uses --reasoning-effort, the flag 1.0.88 documents. --effort still parses but is gone from --help.
  • Claude: the login hint is now claude auth login.

Changed

  • Codex catalogue, from codex debug models: gpt-6-astra (the new default), sol, terra, luna, gpt-daybreak-blue-latest (depends on the account), and gpt-5.5 (retiring 2026-10-14). gpt-5.4 and gpt-5.4-mini are removed.
  • Grok catalogue, from grok models: grok-4.7 (the new default), grok-4.7-build-fast, 4.6 and 4.5.

Verification

  • Live suite: 42 of 46 passed on the first run. Of the other four:
    • The version check passed after the versions were bumped.
    • a_1m_alias_reports_the_widened_window passed after the fix above.
    • codex_interrupts_an_orphaned_active_turn and codex_reads_a_long_idle_session_without_starting_a_turn need prepared fixtures (AGENT_ABSTRACTION_INTERRUPT_SESSION and AGENT_ABSTRACTION_IDLE_SESSION). They were not set, so neither test exercised any code.
  • Copilot mid-run update: Copilot updated itself to 1.0.88 during the work. Its four live tests and the version check were re-run on 1.0.88 and passed.
  • Offline suites: 232 unit tests and 7 process tests pass, and clippy -D warnings is clean.

Not changed yet (each needs a targeted live probe first)

  • Claude: CLAUDE_CODE_OAUTH_TOKEN looks like the variable that setup-token credentials use, and EnvPolicy::Minimal drops it.
  • Claude: rate_limit_event now declares an optional utilization field.
  • Claude: the new --system-prompt-snapshot (on by default) may make --resume ignore a changed --append-system-prompt, which would be a silent downgrade.
  • Codex: codex exec fork now exists headless, and the crate could support fork. For now it stays Unsupported, with the comment corrected.
  • Grok: the binary names effort values none and max, but they were not claimed without a live check.

meh added 5 commits September 23, 2026 22:14
The notification is { error: { message }, threadId, turnId, willRetry }. The
crate read params.message and always got nothing, and it ended the turn as an
error even when codex said it would retry, failing runs that went on to
succeed. The message is read from error.message, and a retried error is left
alone.
--output-format json has no init record, so a run's model name is often unknown,
and claude lists its Haiku helper beside the run's model when the helper ran.
With two entries and no name the [1m] window came back unknown, which the live
suite caught on 2.1.267. The top-level usage is the run's own model: the entry
whose four counts equal it is chosen, only when exactly one does, and anything
else is still reported as unknown.
codex debug models on 0.154.0 lists gpt-6-astra first (the new default), drops
gpt-5.4 and gpt-5.4-mini, adds the account-dependent gpt-daybreak-blue-latest,
and marks gpt-5.5 as retiring on 2026-10-14. grok models on 1.0.40 lists
grok-4.7 (the new default) and grok-4.7-build-fast ahead of 4.6 and 4.5. Grok's
effort list stays as verified: none and max appear only as a string in the
binary.
Checked against claude 2.1.267, codex-cli 0.154.0, Copilot CLI 1.0.88 and
grok 1.0.40, flag by flag against --help, then with the live suite.

- codex exec resume refuses --add-dir, so a resumed run with extra
  directories failed before the agent started. It now carries them as
  -c sandbox_workspace_write.writable_roots=[...], TOML-quoted; a fresh exec
  still takes the flag.
- Copilot documents --reasoning-effort in 1.0.88; --effort still parses but is
  gone from --help, so the documented flag is passed.
- Claude's login hint is `claude auth login`, the subcommand 2.1.267 documents,
  and the quoted stream-json error matches the new wording.
- The comment claiming codex fork is TUI-only is corrected: `codex exec fork`
  exists, but this crate does not drive it, so fork stays Unsupported.
- verified_version moves to the four versions checked.
@pathscale
pathscale merged commit 01b05ea into master Sep 23, 2026
1 check passed
@pathscale
pathscale deleted the fix/verify-current-clis branch September 23, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant