Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,13 @@ env:
jobs:
check:
name: fmt, clippy, test
# One GitHub-hosted runner for the complete decision. Audit used to be a
# second VM that repeated checkout, toolchain setup, and dependency
# resolution. Pin the image so an upstream `latest` change cannot move the
# toolchain underneath an otherwise unchanged pull request.
runs-on: ubuntu-24.04
# One runner for the complete decision. Audit used to be a second VM that
# repeated checkout, toolchain setup, and dependency resolution.
# `-2` is deliberate: this job finishes in well under a minute, and
# Ubicloud bills by vCPU-minute, so a larger runner costs more for no
# wall-clock gain. The repository variable keeps a pool switch operational
# rather than requiring another workflow change.
runs-on: ${{ vars.UBICLOUD_RUNNER || 'ubicloud-standard-2-arm' }}
steps:
- uses: actions/checkout@v4

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ env:
jobs:
publish:
name: publish to crates.io
runs-on: ubuntu-24.04
runs-on: ${{ vars.UBICLOUD_RUNNER || 'ubicloud-standard-2-arm' }}
permissions:
# For pushing the version tag.
contents: write
Expand Down
14 changes: 9 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,15 @@ Copilot CLIs headlessly behind one API. Consumed as a direct dependency by the

## CI runners

Use the explicitly pinned GitHub-hosted `ubuntu-24.04` image. Keep the complete CI
decision in one job so checkout, toolchain setup, dependency resolution, and runner
startup happen once per pull request. Do not add an external runner dependency without
an operational fallback: unavailable third-party capacity must not leave releases queued
indefinitely.
Use the configurable Ubicloud runner in both CI and publishing. The default is
`ubicloud-standard-2-arm`: this pure Rust crate finishes in well under a minute, and a
larger runner adds billed vCPU-minutes without shortening the decision. Keep the complete
CI decision in one job so checkout, toolchain setup, dependency resolution, and runner
startup happen once per pull request.

Ubicloud must be enabled for the repository before its runners can pick up jobs. Change
the `UBICLOUD_RUNNER` repository variable when capacity moves rather than editing workflow
files or silently falling back to GitHub-hosted minutes.

## Build & run

Expand Down
35 changes: 33 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,46 @@ appear in a patch rather than inflating the version toward 1.0 on a crate still
shape. **Where that happens the entry says so at the top**, because a version number that
under-signals is only acceptable if the changelog over-signals to compensate.

## 0.4.15

### Fixed

- **Codex terminal usage now includes every model call in a tool-heavy turn.**
Interactive token updates report disjoint calls. Their additive fields are
accumulated while context-shaped fields remain latest-wins, so the terminal
outcome no longer collapses to only the final call.
- The verified Codex boundary is now CLI `0.146.0`. Visible models marked
`supported_in_api: false`, including inline-only
`gpt-5.3-codex-spark`, are excluded from the runnable catalogue.
- Copilot's flag mapping is verified against CLI `1.0.78`.

## 0.4.14

### Added

- **Named sessions now have cross-process concurrency control.** A run holds a
non-blocking OS lease across the full read-run-commit cycle. A second run on
the same project and session returns `Error::SessionBusy`, which is transient,
instead of forking the provider conversation and silently losing one binding.
The lease is released by the kernel if its holder is killed.

## 0.4.13

### Fixed

- **Writable Codex runs retain the git repository safety check.**
`--skip-git-repo-check` is now limited to ReadOnly and Plan, whose sandbox
cannot edit files. Edit, Auto, and Bypass no longer waive Codex's guard
against changes without a version-control recovery path.

## 0.4.12

### Fixed

- **Codex Auto runs can use GitHub without an approval round trip.** The Codex
workspace sandbox now enables network access for Auto while retaining its
configured writable roots. Edit remains offline and Ask remains interactive.
- Release and pull request checks now use a pinned GitHub-hosted runner so an
unavailable external runner pool cannot leave a release queued indefinitely.
- Release and pull request checks use one configurable Ubicloud runner.

## 0.4.10

Expand Down
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "agent-abstraction"
version = "0.4.12"
version = "0.4.15"
edition = "2024"
# The floor edition 2024 requires, and where the strictest dependencies (uuid,
# getrandom) sit. Derived from the dependency graph rather than compile-tested.
Expand Down Expand Up @@ -39,6 +39,7 @@ tokio = { version = "1", features = ["process", "io-util", "sync", "time", "rt",
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
fs2 = "0.4.3"
# Session ids are caller-minted UUIDv4 (`claude --session-id`, and Copilot's
# handle, which the CLI never prints). v4 only, since these are opaque handles and not
# sort keys, so the v7 timestamp would leak wall-clock into a stable id.
Expand Down
21 changes: 14 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ println!("{:?}", outcome.usage.cost_usd);

## What each agent can actually do

Verified live, against `claude 2.1.205`, `codex-cli 0.145.0` and `GitHub Copilot CLI 1.0.75`
Verified live, against `claude 2.1.205`, `codex-cli 0.146.0` and `GitHub Copilot CLI 1.0.78`
and not inferred from documentation.

| | session id | fork | events | system prompt | resume flag |
Expand Down Expand Up @@ -129,6 +129,13 @@ in two checkouts never collides, and written through a temp file and a rename so
concurrent reader never sees a half-written record. A corrupt record reads as absent: the
next turn opens a fresh conversation rather than failing over a cache nobody asked about.

One named session can have only one run at a time, across processes. `stream()` takes a
non-blocking OS lease for the full read-run-commit cycle and returns `Error::SessionBusy`
when another holder is active. Rebuild or retry the request after that run settles; the
binding is re-read under the lease, so a request prepared while the prior run was active
cannot resume a stale token. The kernel releases the lease if its holder is killed, so a
crashed host cannot deadlock the session.

Names are percent-encoded into a single path segment, which is **injective**: two different
names can never land on the same file. That matters more than it sounds, because the failure
mode of a lossy scheme is silent, not loud. Folding unsafe characters to `-` would map
Expand Down Expand Up @@ -240,7 +247,7 @@ assert_eq!(outcome.structured.unwrap()["name"], "Alice");

The delivery differs and is hidden: Claude takes the schema inline and reports the value in
its own field, Codex reads it from a file this crate writes and removes, and returns the
value as its answer text. **Copilot 1.0.75 has no schema support at all**, so asking is an
value as its answer text. **Copilot 1.0.78 has no schema support at all**, so asking is an
`Error::Unsupported` rather than prose dressed up as data.

**Write schemas strictly.** Codex sends yours to OpenAI's structured-output API, which
Expand Down Expand Up @@ -534,10 +541,10 @@ own variables do not reach the child.

## Gotchas worth knowing

- **`codex exec` refuses to run outside a git repository.** This crate always passes
`--skip-git-repo-check`, so it runs anywhere. That check exists to stop an agent editing
files with no way to undo them; the sandbox is the real containment here, and it defaults
to `read-only`.
- **`codex exec` refuses writable runs outside a git repository.** This crate passes
`--skip-git-repo-check` only for `ReadOnly` and `Plan`, where the sandbox prevents edits.
`Edit`, `Auto`, and `Bypass` retain Codex's guard because a change outside version control
may have no recovery path.
- **`codex exec resume` does not accept `--sandbox`.** It is a different option set from
`codex exec` and rejects the flag outright, so the permission posture is applied as
`-c sandbox_mode=...` on the resume path. Only a multi-turn run reveals this: every
Expand Down Expand Up @@ -766,7 +773,7 @@ A Rust port of [nickderobertis/oneharness](https://github.com/nickderobertis/one

Some findings did not survive re-verification against the current CLIs. oneharness models
Copilot as having no headless session id and no event stream (`session_formats: &[]`,
`events_format: None`); Copilot 1.0.75 has both. Where this crate and oneharness disagree,
`events_format: None`); Copilot 1.0.78 has both. Where this crate and oneharness disagree,
this crate matches what the CLI does today.

## License
Expand Down
75 changes: 44 additions & 31 deletions src/agent.rs
Original file line number Diff line number Diff line change
Expand Up @@ -388,10 +388,10 @@ impl Agent {
let (major, minor, patch) = match self {
// `claude --version` -> "2.1.212 (Claude Code)"
Agent::Claude => (2, 1, 212),
// `codex --version` -> "codex-cli 0.145.0"
Agent::Codex => (0, 145, 0),
// `copilot --version` -> "GitHub Copilot CLI 1.0.75."
Agent::Copilot => (1, 0, 75),
// `codex --version` -> "codex-cli 0.146.0"
Agent::Codex => (0, 146, 0),
// `copilot --version` -> "GitHub Copilot CLI 1.0.78."
Agent::Copilot => (1, 0, 78),
};
crate::Version {
major,
Expand Down Expand Up @@ -536,7 +536,7 @@ impl Agent {
live_follow_up: true,
approvals: true,
},
// Verified against Copilot CLI 1.0.75: `--session-id <uuid>` both
// Verified against Copilot CLI 1.0.78: `--session-id <uuid>` both
// mints a new session and resumes an existing one (one flag, both
// directions), and `--output-format json` is a JSONL event stream.
// There is no headless fork.
Expand All @@ -545,7 +545,7 @@ impl Agent {
fork: false,
events: true,
native_system: false,
// Copilot 1.0.75 exposes no schema flag at all.
// Copilot 1.0.78 exposes no schema flag at all.
schema: SchemaSupport::None,
commands: false,
live_follow_up: false,
Expand Down Expand Up @@ -643,7 +643,7 @@ impl Agent {
/// # Errors
/// [`Error::Unsupported`] if the plan needs a capability this agent lacks.
pub(crate) fn typed_argv(self, plan: &Plan) -> Result<Vec<Arg>> {
// Codex app-server supplies an approval callback. Copilot CLI 1.0.75
// Codex app-server supplies an approval callback. Copilot CLI 1.0.78
// needs `--allow-all-tools` to run headlessly at all and gates only
// through `--deny-tool`. A run that quietly never asked would be the
// worst outcome here, since a caller would read silence as "nothing
Expand All @@ -655,7 +655,7 @@ impl Agent {
what: "routing tool approvals to the caller",
});
}
// Codex app-server accepts `turn/steer`. Copilot CLI 1.0.75 has no
// Codex app-server accepts `turn/steer`. Copilot CLI 1.0.78 has no
// structured input stream and cannot take a second message mid-turn.
if plan.duplex && !caps.live_follow_up {
return Err(Error::Unsupported {
Expand Down Expand Up @@ -931,17 +931,18 @@ fn argv_codex(plan: &Plan) -> Vec<Arg> {
.arg_sensitive(id.clone(), Sensitivity::SessionId);
}

// `codex exec` aborts outside a git repository unless told not to. That
// check guards against an agent editing files with no way to undo them, but
// this crate is embedded in hosts that legitimately run against scratch
// directories, worktrees and review checkouts, and a hard abort there is
// useless to them. The real containment is the sandbox below, which is
// `read-only` by default, so nothing is unrecoverable regardless.
a.bare("--skip-git-repo-check");
// `codex exec` aborts outside a git repository unless told not to. Waiving
// that check is safe only while the sandbox cannot write: scratch
// directories and review exports remain readable, while Edit, Auto, and
// Bypass retain Codex's guard against changes with no version-control
// recovery path.
if matches!(plan.permission, Permission::ReadOnly | Permission::Plan) {
a.bare("--skip-git-repo-check");
}

// `codex exec` takes `--sandbox`, but `codex exec resume` does **not**: it
// rejects the flag outright and takes the same setting as a `-c` config
// override instead. Verified against codex-cli 0.145.0, where passing
// override instead. Verified against codex-cli 0.146.0, where passing
// `--sandbox` to a resume fails with "unexpected argument '--sandbox'".
// Dropping the sandbox on resume would silently run a continued turn under a
// different posture than the caller asked for.
Expand All @@ -960,13 +961,13 @@ fn argv_codex(plan: &Plan) -> Vec<Arg> {
};

a.opt("--model", plan.model.as_ref());
// Verified against codex-cli 0.145.0: `codex exec` has no effort flag, it
// Verified against codex-cli 0.146.0: `codex exec` has no effort flag, it
// is a config override, and `--strict-config` accepts this key. A bad value
// is refused by the provider with its own enum rather than by the CLI.
if let Some(effort) = plan.effort.as_ref() {
a.pair("-c", format!("model_reasoning_effort={effort}"));
}
// Verified against codex-cli 0.145.0. Options remain options after the
// Verified against codex-cli 0.146.0. Options remain options after the
// positional prompt, but keeping roots before it makes the command's
// security posture readable and matches the CLI's help shape.
for dir in &plan.extra_dirs {
Expand Down Expand Up @@ -1000,7 +1001,7 @@ fn argv_codex(plan: &Plan) -> Vec<Arg> {

/// `copilot -p <prompt> --allow-all-tools [...] [--session-id <uuid>]`
///
/// Flags verified against Copilot CLI 1.0.75. Two of its conventions matter:
/// Flags verified against Copilot CLI 1.0.78. Two of its conventions matter:
/// `--allow-all-tools` is *required* for non-interactive mode, and the
/// repeatable tool filters are declared `--allow-tool[=tools...]`, an optional
/// value, which only binds with `=`, never across a space.
Expand Down Expand Up @@ -1031,7 +1032,7 @@ fn argv_copilot(plan: &Plan) -> Vec<Arg> {
};

a.opt("--model", plan.model.as_ref());
// Verified against Copilot CLI 1.0.75: `--effort` is the documented spelling
// Verified against Copilot CLI 1.0.78: `--effort` is the documented spelling
// and `--reasoning-effort` its alias (none, minimal, low, medium, high,
// xhigh, max). A wider set than Claude's, which is why the level is passed
// through rather than mapped to a shared enum.
Expand Down Expand Up @@ -1458,18 +1459,30 @@ mod tests {
}
}

/// `codex exec` aborts outside a git repository. A host embedding this
/// crate runs against scratch dirs and review checkouts, so the check is
/// waived on every invocation; the sandbox is what actually contains a run.
/// Read-only runs can inspect scratch dirs and review exports safely. A
/// writable posture keeps Codex's repository guard, since there may be no
/// way to undo a change outside version control.
#[test]
fn codex_always_waives_the_git_repo_check() {
fn codex_waives_the_git_repo_check_only_without_writes() {
for cont in [Continue::New, Continue::Resume("t-1".into())] {
let mut p = plan("codex");
p.cont = cont.clone();
assert!(
argv(Agent::Codex, &p).contains(&"--skip-git-repo-check".to_string()),
"{cont:?} must still run outside a repo"
);
for permission in [Permission::ReadOnly, Permission::Plan] {
let mut p = plan("codex");
p.cont = cont.clone();
p.permission = permission;
assert!(
argv(Agent::Codex, &p).contains(&"--skip-git-repo-check".to_string()),
"{cont:?} {permission:?} must still read outside a repo"
);
}
for permission in [Permission::Edit, Permission::Auto, Permission::Bypass] {
let mut p = plan("codex");
p.cont = cont.clone();
p.permission = permission;
assert!(
!argv(Agent::Codex, &p).contains(&"--skip-git-repo-check".to_string()),
"{cont:?} {permission:?} must keep Codex's repository guard"
);
}
}
}

Expand Down Expand Up @@ -1518,7 +1531,7 @@ mod tests {
assert!(!a.iter().any(|arg| arg.contains("\"type\"")));
}

/// Copilot 1.0.75 has no schema flag, and a prose answer presented as data
/// Copilot 1.0.78 has no schema flag, and a prose answer presented as data
/// is exactly the silent downgrade this crate refuses elsewhere.
#[test]
fn copilot_refuses_a_schema_rather_than_answering_in_prose() {
Expand Down
31 changes: 30 additions & 1 deletion src/codex_app_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,14 @@ impl Protocol {
}
"thread/tokenUsage/updated" => {
if let Some(usage) = usage(&params) {
self.terminal.usage = usage;
// `last` is one model call, not the whole interactive
// turn. Tool-heavy turns receive one update after every
// call; replacing here made the live stream correctly add
// 1.7M processed tokens while the terminal outcome fell
// back to only its final 226k call. The snapshots are
// disjoint billing traffic, so accumulate their additive
// fields while keeping context-shaped fields latest.
self.terminal.usage.accumulate(&usage);
step.events.push(Event::Usage(usage));
}
}
Expand Down Expand Up @@ -651,6 +658,28 @@ mod tests {
assert_eq!(usage.context_window, Some(258_400));
}

#[test]
fn codex_terminal_usage_accumulates_every_model_call_in_the_turn() {
let mut protocol = Protocol::new(request());
for (input, cached, output) in [(206_011, 188_160, 321), (206_692, 204_544, 285)] {
protocol.push(&json!({
"method": "thread/tokenUsage/updated",
"params": {"tokenUsage": {"last": {
"inputTokens": input,
"cachedInputTokens": cached,
"outputTokens": output,
"reasoningOutputTokens": 0
}, "modelContextWindow": 997_500}},
}));
}

assert_eq!(protocol.terminal.usage.input_tokens, Some(19_999));
assert_eq!(protocol.terminal.usage.cache_read_tokens, Some(392_704));
assert_eq!(protocol.terminal.usage.output_tokens, Some(606));
assert_eq!(protocol.terminal.usage.context_tokens, Some(206_692));
assert_eq!(protocol.terminal.usage.context_window, Some(997_500));
}

#[test]
fn completed_agent_messages_preserve_their_boundary() {
let mut protocol = Protocol::new(request());
Expand Down
Loading
Loading