feat(demo): Pydantic title investigation [DO NOT MERGE] - #13671
Draft
SunkenInTime wants to merge 22 commits into
Draft
SunkenInTime wants to merge 22 commits into
SunkenInTime wants to merge 22 commits into
Conversation
Provider turns become invoke_agent spans with execute_tool and chat children, derived from canonical runtime events so all adapters share one mapping. Spans go only to the configured OTLP exporter, with pending spans for in-progress runs and content capture behind the standard OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adapters now emit model.response.completed with each main-agent response's request start, first chunk, end, tool calls, and exact usage. Claude derives the start from Claude Code's own ttft_ms; Codex enables experimentalRawEvents and ends responses at rawResponse/completed, so chat spans no longer include tool time. Claude tool spans open when the requesting response ends instead of when the call starts streaming. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex runs one model tool call (an exec script) as several command items, so command ids never matched the model's call ids. The Codex adapter now reports each model call and its recorded output as model.tool_call.started/completed, and telemetry nests the commands under the call that ran them. Commands whose process outlives the call end when the call returns and say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
knip flagged the exports; only the recorder is used outside the module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sensitive keys now hide objects and arrays too, matching Logfire's scrubber. Prompts whose turn never starts expire after five minutes and are cleared when the session exits, as are per-thread session facts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Record the prompt after sendTurn returns, keyed by its turn id, so a still-queued turn.completed cannot hand it to the previous turn. - Ignore late completions of Codex executions already ended with their call, and count a failed call once however many commands failed. - An invalid content-capture setting logs a warning instead of failing the provider runtime layer. - Import the agent telemetry and provider service modules as namespaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fast turn can start and finish before sendTurn returns, so prompts are noted before the send (in arrival order, never handed to a turn that was already running) and bound to the returned turn id afterwards. Streamed text buffers stop at the export limit, and token-bearing keys are scrubbed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ound 4 - A starting turn takes only sends bound to it. A turn that finishes while its thread has an unbound send stays open until sendTurn binds the send (or reports failure), so neither a fast turn nor a Claude background turn gets the wrong prompt. - Subagents nest under the Task call that is still streaming. - Export turn-level cache writes, and label responses after a reroute with the new model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Background tasks (shells, monitors, plans) are classified in the recorder, since ingestion stamps agentKind later, and are not exported as subagents. - Codex child agents that end through task.updated close with their real status. - Subagent descriptions and failure summaries stay out of spans unless content capture is on. - A Claude tool that finished while its response was still streaming is labeled t3.tool.duration_upper_bound, since its true start is unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pletions A subagent still running when its turn ends (a Claude background agent) now ends with its own completion instead of being marked interrupted; session exit or shutdown still closes it. Only Codex child agents close on a terminal task.updated, so Claude's later task.completed keeps its summary and usage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
vouch:trusted
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Demo-only draft. DO NOT MERGE. This branch deliberately reproduces a title-generation regression for a Pydantic sponsor demo.
The first message asks for offline search, but its generated title is about label colors. Three constructed chats exercise T3 Code's real title prompt builder and Codex adapter. An investigator uses Logfire MCP to inspect the supplied input, makes a focused repair, and checks the same cases with Pydantic Evals.
Open the public setup and recording handoff. It includes a copyable setup prompt and a downloadable pack with the constructed corpus and reference results. The page requires no login. Recipients use their own Codex and Logfire accounts; no personal T3 database or credentials are distributed.
The handoff pins
b83de9d84a86a687642f3514094ae2480d816a13. This updates the earlier snapshot with native Logfire Agent Run conversation formatting, live prompt and assistant-message records, nested MCP arguments/results, and bounded desktop startup requests. The intentional bad-title baseline remains in the committed source.Use the pinned repository guide, or copy the handoff's setup prompt into a coding agent. It prepares an isolated recording desktop, runs one three-case baseline against the recipient's Logfire project, verifies the hosted evidence, and stops before the investigation. The recorded reference baseline passes 1/3 subject checks; a previous repair rehearsal reached 3/3. Fresh model results can vary.
Validation: a real Astra run was inspected in Logfire's native Agent Run view, including messages visible before completion and MCP requests/results. The macOS recording desktop was rebuilt and visually checked. The 26 telemetry tests, targeted lint, server typecheck, desktop build and four isolated seeder tests passed. The public handoff and its copy buttons were browser-checked; unauthenticated downloads match the local artifacts. This packaging pass does not claim a new clean-machine rehearsal or cross-platform desktop validation.
Keep this branch unmerged. The constructed failure is not a claim about released T3 Code or a model benchmark. Agent telemetry covers observed current-turn events, not provider-hidden context. An in-progress repair in the maintainer's working checkout is excluded from the pinned snapshot.
Model: GPT-6
Harness: Codex