Skip to content

fix: recognize failed V2 ACP tool results as failures - #3

Open
ranxianglei wants to merge 2 commits into
drexb-ops:masterfrom
ranxianglei:2026-09-17_v2-failed-tool-recognition
Open

ranxianglei wants to merge 2 commits into
drexb-ops:masterfrom
ranxianglei:2026-09-17_v2-failed-tool-recognition

Conversation

@ranxianglei

Copy link
Copy Markdown

Fixes ranxianglei#426

Problem

The pinned @opencode/plugin@2.0.3 Promise adapter has no safe native error channel, so createV2Tool returns resolved results for caught execution failures and the host records them as state.status === "completed" even when the content says ACP compress failed: .... ACP's internal V2 projection preserved that completed status, so:

  • messageHasCompress (lib/messages/query.ts) treated a failed compression as success;
  • nudge logic (lib/messages/inject/inject.ts) advanced lastPerMessageNudgeTokens / set compressBaselineSet after a failed compression;
  • cold-start reconstruction (lib/state/rebuild.ts) replayed the failed call and rebuilt a block that never existed;
  • hideFailedCompressCalls never hid the failed call (it only matches status === "error").

The installed E2E fake-provider classifier also missed these failure texts (ACP .* execution failed does not match ACP compress failed:).

Fix

  • errorResult() in lib/v2/tools.ts adds explicit acpFailed: true metadata to every resolved error result (additive — existing per-reason flags unchanged).
  • New pure module lib/v2/projection/acp-failure.ts: canonical ACP tool-name list, metadata key, position-0-anchored failure-text pattern, predicates — shared by the projection and the E2E fake provider so they cannot drift.
  • toolState() projects a host-completed result for an ACP tool to an internal status: "error" when explicit failure metadata is present or the output matches the historical failure-text pattern (pre-fix host history). It exposes no output field, so provider-owned lowered results are never rewritten: origin correlation keeps the lowered output untouched and the patcher's status-change guard sees identical states on both sides.
  • inspectToolResults in scripts/e2e/fake-llm-server.ts classifies ACP results through the shared anchored pattern (legacy regex retained for non-ACP tools).
  • installed-v2.ts nudge-growth stage now asserts the observed compress results are genuinely completed, not merely present.

No nudge code changes were needed: messageHasCompressAttempt is status-agnostic (failed attempts still reset pending-nudge state), while messageHasCompress requires completed — once the projection emits error for failures, success baselines stop advancing and V2 aligns with V1 semantics exactly.

Verification

  • 18 new tests in tests/v2-failed-tool-recognition.test.ts (pure module, projection shapes, cold rebuild, warm multi-turn nudge baselines with preserveRecentMessages: 20); verified they fail with the projection fix reverted (bug-detection check per AGENTS.md §5.7.3).
  • Full suite: 1428/1429 pass. The single failure is tests/soft-block.test.ts, which hardcodes mkdirSync('/tmp/...') at import time and crashes on this sandbox's read-only /tmp mount — environmental, unrelated to this diff (passes in CI).
  • Typecheck, build, and Prettier checks pass on the changed files.
  • V1 behavior unchanged: recognition is confined to the V2 projection boundary; successful ACP outputs, non-ACP tools, native host error states, and persisted state formats are all untouched.

The pinned @opencode/plugin@2.0.3 Promise adapter has no safe native
error channel, so createV2Tool returns resolved results for caught
failures and the host records them as status completed. ACP's internal
V2 projection preserved that completed status, so messageHasCompress,
nudge success baselines, and cold-start reconstruction treated failed
compressions as successes; the installed E2E fake provider classifier
missed the same failure texts.

- errorResult() now adds acpFailed:true metadata to every resolved
  error result (additive; existing per-reason flags unchanged)
- new pure module lib/v2/projection/acp-failure.ts holds the canonical
  tool-name list, metadata key, anchored failure-text pattern, and
  predicates shared by the projection and the E2E fake provider
- toolState projects a completed host state for an ACP tool to an
  internal status:error when explicit failure metadata is present or
  the output matches the historical failure-text pattern; it exposes
  no output field, so provider-owned lowered results are never
  rewritten (origin correlation + patcher invariants hold)
- fake-llm-server inspectToolResults classifies ACP results via the
  shared anchored pattern; legacy regex retained for other tools
- installed-v2 nudge-growth asserts genuine completed status for both
  compress results
- tests: 18 new tests (pure module, projection, cold rebuild, warm
  nudge baselines with preserveRecentMessages:20); verified they fail
  with the projection fix reverted
- rename misleading isAcpFailedToolOutput param neutralizedOutput -> outputText
- strengthen warm-nudge control assertion: success baseline must advance
  past its seeded value, not merely become defined
- add two full-projection integration tests driving host-shaped completed
  records through normalizeV2ProjectedHistory (internal part status error
  without output; origin.normalizedOutput stays undefined so the patcher
  can never rewrite provider-owned output)
- document shared-tools soft-string failure limitation (DESIGN.md section 6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V2: resolved ACP error results are mistaken for successful compression

1 participant