Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,49 @@ appear in a patch rather than inflating the version toward 1.0 on a crate still
shape. **Where that happens the entry says so at the top**, because a version number that
under-signals is only acceptable if the changelog over-signals to compensate.

## 0.5.2

**Behaviour change in a patch.** A run on a model the catalogue marks retired now fails
with the new `Error::RetiredModel` before anything is spawned, where before it ran. `Model` also
gains a public `retired` field (additive: `Model` is `#[non_exhaustive]`).

Model catalogues re-checked against the installed CLIs: claude 2.1.280 and codex-cli
0.156.1. Every Claude entry was run once with `--model` and answered under the id it
reports; the Codex list is what `codex debug models` returns. The live Codex suite passed
on 0.156.1 apart from the two tests that need a session fixture.

### Added

- **Claude Opus 5.5:** `claude-opus-5-5` and `claude-opus-5-5[1m]`, both 1M. On 2.1.280
`opus` and `default` resolve to `claude-opus-5-5`, and `opus[1m]` to
`claude-opus-5-5[1m]`. 2.1.267 refused the id as `[claude-code:unrecognized_model]`.
- **Retired models stay in the catalogue.** A model a CLI stops offering is kept with
`Model::retired` set (`Retired { since, replacement, reason }`) instead of being deleted,
because a host that stored the id would otherwise see it vanish from its picker while
its runs kept using it. `Model::is_retired()` flags it, a retired entry is never
`is_default`, and a request naming one fails with `Error::RetiredModel` carrying the model,
the reason and the replacement. Ids the catalogue does not know still pass through.
`discover_models` appends the compiled-in retired entries after the discovered ones.
Retired: `claude-fable-5` (claude 2.1.267, answers as `claude-opus-4-8`; use
`claude-fable-5-1`), and Codex `gpt-5.4` and `gpt-5.4-mini` (codex-cli 0.154.0, no longer
listed; use `gpt-6-astra`).

### Changed

- **Codex catalogue** follows what 0.156.1 reports: `gpt-6-sol` and `gpt-6-luna` are new,
between `gpt-6-astra` (still the default) and `gpt-5.6-sol`. `gpt-6-luna` stops at `max`;
`gpt-6-sol` goes to `ultra`.
- **`verified_version`** is claude 2.1.280 and codex 0.156.1, the releases `Probe` now
compares against. `claude --help` on 2.1.280 differs from 2.1.267 only in the wording of
`--bare` and `--safe-mode`; no flag this crate passes changed.
- **Claude catalogue:** `claude-fable-5-1` is added and `claude-fable-5` is retired. Both
`fable` and `best` now resolve to `claude-fable-5-1`, and a run on `claude-fable-5`
produced no output of its own, with the answer coming from `claude-opus-4-8`.
`claude-haiku-4-5-20251001`, the id the `haiku` alias reports, is added beside
`claude-haiku-4-5`, which still answers under its own name and stays live.
- **`claude-opus-5` is 1M natively** since 2.1.267. It reported a 200k window on 2.1.212;
it now reports 1,000,000 without the `[1m]` suffix.

## 0.5.1

Re-verified against the installed CLIs: claude 2.1.267, codex-cli 0.154.0, GitHub Copilot
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "agent-abstraction"
version = "0.5.1"
version = "0.5.2"
edition = "2024"
# The floor edition 2024 requires, and where the strictest dependencies (uuid,
# getrandom) sit. Derived from the dependency graph rather than compile-tested.
Expand Down
52 changes: 40 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Every entry point that spawns a CLI (`run`, `stream`, `interrupt`, `Probe::run`,

## What each agent can actually do

Verified live, against `claude 2.1.267`, `codex-cli 0.154.0`, `GitHub Copilot CLI 1.0.88` and
Verified live, against `claude 2.1.280`, `codex-cli 0.156.1`, `GitHub Copilot CLI 1.0.88` and
`grok 1.0.40`, and not inferred from documentation.

| | session id | fork | events | system prompt | resume flag |
Expand Down Expand Up @@ -167,10 +167,28 @@ for model in Agent::Claude.models() {
}
```

The list is **advisory and never enforced**. `Request::model` takes any string and nothing
here checks it, so a model released this morning is not blocked by a list compiled last
month. A model the account cannot reach comes back as `Error::AgentError` with the
provider's own status and wording.
The list is **advisory, with one exception**. `Request::model` takes any string and an id
the list does not know passes straight through, so a model released this morning is not
blocked by a list compiled last month. A model the account cannot reach comes back as
`Error::AgentError` with the provider's own status and wording.

The exception is a **retired** entry. A model a CLI stops offering is kept in the list with
`Model::retired` set (`since`, `reason`, and a `replacement` where there is one) instead of
being deleted, because a host that stored the id would otherwise watch it vanish from its
picker while its runs kept using it. `Model::is_retired()` lets a picker flag it; a retired
entry is never `is_default`. A request naming one is refused with `Error::RetiredModel`
before anything is spawned. It carries the model and its `Retired`, so a host can offer the
replacement, and the user picks another model:

| Agent | Retired id | Since | Replacement | Why |
|---|---|---|---|---|
| Claude | `claude-fable-5` | claude 2.1.267 | `claude-fable-5-1` | Still accepted, but the answer comes from `claude-opus-4-8` |
| Codex | `gpt-5.4` | codex-cli 0.154.0 | `gpt-6-astra` | No longer listed by `codex debug models` |
| Codex | `gpt-5.4-mini` | codex-cli 0.154.0 | `gpt-6-astra` | No longer listed by `codex debug models` |

Retirement is per agent: Copilot's own `gpt-5.4` and `claude-fable-5` entries are untouched.
`claude-haiku-4-5` is not retired; it still answers under its own name and sits beside the
dated `claude-haiku-4-5-20251001` that the `haiku` alias reports.

That distinction matters more than it sounds, because **a catalogue is not an entitlement**.
Copilot's picker lists twenty-three models and a Free plan permits exactly one:
Expand All @@ -187,7 +205,8 @@ the account actually allows. `Model::is_default` marks the safe pre-selection.
Aliases and pinned ids are both carried, and `Model::kind` tells them apart, because they
do not always agree. On claude 2.1.212, `--model opus` reported `claude-opus-4-8` while
`--model claude-opus-5` reported `claude-opus-5`, even though that release's own notes call
Opus 5 the default Opus model. An alias is whatever the account resolves it to.
Opus 5 the default Opus model. On 2.1.280 `opus` reports `claude-opus-5-5`
and `fable` reports `claude-fable-5-1`. An alias is whatever the account resolves it to.

### Reasoning effort

Expand All @@ -199,8 +218,8 @@ let request = Request::new(Agent::Codex, prompt).model("gpt-5.6-sol").effort("ul

Passed through verbatim, like the model, because the sets are not interchangeable: Claude
documents five levels, Copilot seven, and Codex varies them per model, offering `ultra` on
its two frontier models and not on the rest. Delivered as `--effort` on Claude and Copilot,
and as `-c model_reasoning_effort=<level>` on Codex, which has no flag for it.
its larger models and not on the rest. Delivered as `--effort` on Claude, as
`--reasoning-effort` on Copilot and Grok, and as `-c model_reasoning_effort=<level>` on Codex, which has no flag for it.

Support is not uniform even within one agent. Copilot's `auto` **exits 1** rather than
ignoring the flag:
Expand All @@ -224,6 +243,11 @@ returning the compiled list: both enumerate models only in an interactive picker
caller asking for discovery is asking for freshness. `Agent::models_verified()` records how
each compiled list was established and against which release.

Discovery still appends the compiled list's retired entries after the discovered ones, since
a CLI lists only what it offers today and a picker has to show and flag a stored id that has
gone. An id the CLI lists that the compiled list marks retired is flagged in place, not
repeated.

### Disabling thinking

`Request::thinking(false)` turns the model's reasoning off for a run; left unset the agent
Expand Down Expand Up @@ -571,11 +595,15 @@ own variables do not reach the child.
cleanly, reports `subtype: "success"`, and puts "There's an issue with the selected model"
where the answer belongs. Codex does the same and wraps the upstream body in a JSON string.
Both come back as `Error::AgentError`; see below.
- **`claude-opus-5` defaults to a 200k window while every other 5-series model is 1M.**
Verified by running each id: `claude-sonnet-5` and `claude-fable-5` report a 1,000,000
token window natively, `claude-opus-5` reports 200,000 and needs the `[1m]` suffix
(`claude-opus-5[1m]`) to widen. The catalogue carries both forms.
- **`claude-opus-5`'s window depends on the CLI release.** On 2.1.212 it reported 200,000
and needed the `[1m]` suffix (`claude-opus-5[1m]`) to widen, while `claude-sonnet-5` and
`claude-fable-5` reported 1,000,000 natively. On 2.1.267 plain `claude-opus-5` reports
1,000,000 too. The catalogue carries both forms. Opus 5.5 (2.1.280) is 1M plain and suffixed.
- **Claude's `stream-json` requires `--verbose`**, or it refuses to start. Handled.
- **Claude accepts an unknown `--effort` and runs at the default.** On 2.1.280 `--effort
ultra` prints a warning to stderr and exits 0. `Request::effort` passes the level through
unvalidated, so an effort outside `Model::efforts` is a silent downgrade on Claude today.
Pick efforts from the catalogue.
- **Large prompts move to stdin automatically** above 128 KiB, so a long prompt never fails
with `E2BIG`.

Expand Down
8 changes: 4 additions & 4 deletions src/agent.rs
Original file line number Diff line number Diff line change
Expand Up @@ -397,10 +397,10 @@ impl Agent {
#[must_use]
pub fn verified_version(self) -> crate::Version {
let (major, minor, patch) = match self {
// `claude --version` -> "2.1.267 (Claude Code)"
Agent::Claude => (2, 1, 267),
// `codex --version` -> "codex-cli 0.154.0"
Agent::Codex => (0, 154, 0),
// `claude --version` -> "2.1.280 (Claude Code)"
Agent::Claude => (2, 1, 280),
// `codex --version` -> "codex-cli 0.156.1"
Agent::Codex => (0, 156, 1),
// `copilot --version` -> "GitHub Copilot CLI 1.0.88."
Agent::Copilot => (1, 0, 88),
// `grok --version` -> "grok 1.0.40 (eb1a2256660d)"
Expand Down
24 changes: 24 additions & 0 deletions src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,30 @@ pub enum Error {
message: String,
},

/// The request named a model the agent's catalogue marks retired.
///
/// A user state, not a programming mistake: the id was valid once and a
/// host may have stored it. Refused rather than run, because a retired id
/// can still be accepted and answer as a different model. A host shows
/// this as "pick another model", offering [`crate::Retired::replacement`].
#[error(
"{agent} no longer offers `{model}` (retired since {}: {}); {}",
retired.since,
retired.reason,
match &retired.replacement {
Some(replacement) => format!("pick `{replacement}` instead"),
None => "pick another model".to_string(),
}
)]
RetiredModel {
/// The agent that was asked.
agent: Agent,
/// The id the request named.
model: String,
/// Why it was retired, and what replaces it.
retired: crate::Retired,
},

/// The request asked an agent for something it cannot do headlessly:
/// forking on Codex, a named session on Copilot, an event stream on an
/// agent that only prints text.
Expand Down
2 changes: 1 addition & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ pub use auth::{AuthState, AuthStatus};
pub use command::{Command, Commands, Compaction};
pub use error::{Error, Result};
pub use event::{Event, MAX_CAPTURE, MAX_EVENT_BYTES, MAX_LINE, TRUNCATION_MARK};
pub use model::{Kind, Model, Source, Verified};
pub use model::{Kind, Model, Retired, Source, Verified};
pub use outcome::{Outcome, RateLimit, Stop, Usage};
pub use probe::{Probe, Version, VersionStatus};
pub use request::Request;
Expand Down
Loading
Loading