fix(claude_agent_sdk): stop assigning query usage to the final call - #670
Open
Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
Open
fix(claude_agent_sdk): stop assigning query usage to the final call#670Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
Conversation
Abhijeet Prasad (AbhiPrasad)
force-pushed
the
abhi-claude-agent-sdk-token-accounting
branch
from
August 10, 2026 16:23
fb47358 to
77b1058
Compare
ResultMessage usage is cumulative across the query, so attaching it to the
final anthropic.messages.create span misattributes root and subagent tokens
and produces incorrect prompt-cache costs.
The caller's include_partial_messages option now determines where exact
per-request completion usage comes from. Braintrust leaves this option
unchanged because enabling it would expose additional public stream events.
include_partial_messages=True
message_start -> request ID and per-request prompt/cache usage
message_delta -> final per-request completion usage
result -> query metadata only
include_partial_messages=False or omitted
assistant -> request ID and per-request prompt/cache usage
transcript -> final completion usage when an exact match exists
no match -> omit completion_tokens and tokens
result -> query metadata only
Passive hooks keep root and subagent transcript paths separate:
UserPromptSubmit --------> root transcript ------> [root, message ID]
SubagentStop(tool A) ----> transcript A ---------> [tool A, message ID]
SubagentStop(tool B) ----> transcript B ---------> [tool B, message ID]
The agent context and message ID form one lookup key, preventing root and
parallel subagent calls from sharing usage. Transcript recovery searches
backward for the newest valid usage row and retries briefly for write races:
assistant(id=msg_1) -----> save [context, msg_1] and span
query stream ends -------> read context transcript
<----- newest valid usage row
update span [context, msg_1]
missing row -------------> wait 25 ms and retry (maximum: 3 reads)
Result usage remains query metadata only, and aggregate token metrics stay off
the parent task to avoid double-counting. Caller options and yielded messages
remain unchanged.
Anthropic cache metrics now emit either TTL-specific or legacy cache-creation
fields, never both, and prompt totals use the same representation. Sanitized
subprocess cassettes cover partial, transcript, subagent, and fallback paths.
Resolves #564
Abhijeet Prasad (AbhiPrasad)
force-pushed
the
abhi-claude-agent-sdk-token-accounting
branch
from
August 10, 2026 16:46
77b1058 to
efdf298
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
resolves #564
resolves https://linear.app/braintrustdata/issue/SDK-52/claude-agent-sdk-handle-result-logs-main-agent-usage-under-counts
AI Summary
ResultMessage usage is cumulative across the query, so attaching it to the final anthropic.messages.create span misattributes root and subagent tokens and produces incorrect prompt-cache costs.
The caller's include_partial_messages option now determines where exact per-request completion usage comes from. Braintrust leaves this option unchanged because enabling it would expose additional public stream events.
Passive hooks keep root and subagent transcript paths separate:
The agent context and message ID form one lookup key, preventing root and parallel subagent calls from sharing usage. Transcript recovery searches backward for the newest valid usage row and retries briefly for write races:
Result usage remains query metadata only, and aggregate token metrics stay off the parent task to avoid double-counting. Caller options and yielded messages remain unchanged.
Anthropic cache metrics now emit either TTL-specific or legacy cache-creation fields, never both, and prompt totals use the same representation. Sanitized subprocess cassettes cover partial, transcript, subagent, and fallback paths.