Skip to content

fix(claude_agent_sdk): stop assigning query usage to the final call - #670

Open
Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
mainfrom
abhi-claude-agent-sdk-token-accounting
Open

fix(claude_agent_sdk): stop assigning query usage to the final call#670
Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
mainfrom
abhi-claude-agent-sdk-token-accounting

Conversation

@AbhiPrasad

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) commented Aug 10, 2026

Copy link
Copy Markdown
Member

resolves #564
resolves https://linear.app/braintrustdata/issue/SDK-52/claude-agent-sdk-handle-result-logs-main-agent-usage-under-counts

AI Summary

ResultMessage usage is cumulative across the query, so attaching it to the final anthropic.messages.create span misattributes root and subagent tokens and produces incorrect prompt-cache costs.

The caller's include_partial_messages option now determines where exact per-request completion usage comes from. Braintrust leaves this option unchanged because enabling it would expose additional public stream events.

    include_partial_messages=True
      message_start -> request ID and per-request prompt/cache usage
      message_delta -> final per-request completion usage
      result        -> query metadata only

    include_partial_messages=False or omitted
      assistant  -> request ID and per-request prompt/cache usage
      transcript -> final completion usage when an exact match exists
      no match   -> omit completion_tokens and tokens
      result     -> query metadata only

Passive hooks keep root and subagent transcript paths separate:

    UserPromptSubmit --------> root transcript ------> [root, message ID]

    SubagentStop(tool A) ----> transcript A ---------> [tool A, message ID]
    SubagentStop(tool B) ----> transcript B ---------> [tool B, message ID]

The agent context and message ID form one lookup key, preventing root and parallel subagent calls from sharing usage. Transcript recovery searches backward for the newest valid usage row and retries briefly for write races:

    assistant(id=msg_1) -----> save [context, msg_1] and span
    query stream ends -------> read context transcript
                                <----- newest valid usage row
                               update span [context, msg_1]

    missing row -------------> wait 25 ms and retry (maximum: 3 reads)

Result usage remains query metadata only, and aggregate token metrics stay off the parent task to avoid double-counting. Caller options and yielded messages remain unchanged.

Anthropic cache metrics now emit either TTL-specific or legacy cache-creation fields, never both, and prompt totals use the same representation. Sanitized subprocess cassettes cover partial, transcript, subagent, and fallback paths.

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) force-pushed the abhi-claude-agent-sdk-token-accounting branch from fb47358 to 77b1058 Compare August 10, 2026 16:23
ResultMessage usage is cumulative across the query, so attaching it to the
final anthropic.messages.create span misattributes root and subagent tokens
and produces incorrect prompt-cache costs.

The caller's include_partial_messages option now determines where exact
per-request completion usage comes from. Braintrust leaves this option
unchanged because enabling it would expose additional public stream events.

    include_partial_messages=True
      message_start -> request ID and per-request prompt/cache usage
      message_delta -> final per-request completion usage
      result        -> query metadata only

    include_partial_messages=False or omitted
      assistant  -> request ID and per-request prompt/cache usage
      transcript -> final completion usage when an exact match exists
      no match   -> omit completion_tokens and tokens
      result     -> query metadata only

Passive hooks keep root and subagent transcript paths separate:

    UserPromptSubmit --------> root transcript ------> [root, message ID]

    SubagentStop(tool A) ----> transcript A ---------> [tool A, message ID]
    SubagentStop(tool B) ----> transcript B ---------> [tool B, message ID]

The agent context and message ID form one lookup key, preventing root and
parallel subagent calls from sharing usage. Transcript recovery searches
backward for the newest valid usage row and retries briefly for write races:

    assistant(id=msg_1) -----> save [context, msg_1] and span
    query stream ends -------> read context transcript
                                <----- newest valid usage row
                               update span [context, msg_1]

    missing row -------------> wait 25 ms and retry (maximum: 3 reads)

Result usage remains query metadata only, and aggregate token metrics stay off
the parent task to avoid double-counting. Caller options and yielded messages
remain unchanged.

Anthropic cache metrics now emit either TTL-specific or legacy cache-creation
fields, never both, and prompt totals use the same representation. Sanitized
subprocess cassettes cover partial, transcript, subagent, and fallback paths.

Resolves #564
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude Agent SDK: _handle_result logs main-agent usage, under-counts turns with subagents

1 participant