[ROUTE6-1:T3] Durable model-canary attribution across resumes and model switches - #6
Conversation
RESULT ROUTE6-1:T3 — durable model-canary attributionStatus: complete (source + migrations + focused tests + docs; stacked draft PR). CI cannot be observed green on the fork (see Checks).
Relationship to t3code#4Stacked on #4's branch head, not a rewrite. #4 remains the implementation line for the attribution proof; this PR is the narrow follow-on #4 itself named and deferred ("durable per-thread session history … specified but not adopted"). Committed diff is only the new commits on top of Schema / storage decisionA migration was used because existing schema cannot truthfully preserve the required history:
Both registered in Durable history semantics
Experiment metadata semantics
Requested vs actual model handling
Request-identity rule preserved from #4Unsupported request/prompt identity stays Privacy boundaryMetadata only. No raw prompts, responses, source code, tool-call bodies, secrets, or arbitrary user content are stored or logged anywhere in this path. Tests (17 files, 285 passed)Covers the twelve required semantics: (1) one thread DeepSeek→Luna keeps both; (2) resume into a new provider session keeps the earlier one; (3) child OpenCode identity distinct, Typecheck / lint / formatCI
Actual route used for THIS agent sessionopencode-go / Blockers / unknowns
Exact next actionRequest review of (a) the two append-only tables and the upsert-time write path, (b) the |
|
ROUTE6-1:T3 (#6) — corrected #4 head to rebase onto This follow-on is stacked on #4's old head
Compatibility notes for your rebase (your migrations 054/055 and durable-history work were not touched here):
Please rebase after review of #4. No merge. |
|
START ROUTE6-1:R2-T3 - rebase durable attribution onto corrected #4 head
|
T3's single resume cursor lost earlier native sessions on a resume, fork, or model switch, so a thread could not say which models contributed. Add an append-only provider_session_history (migration 054, with a current-cursor backfill) written alongside the runtime upsert, and read it through usageAttributionSources as sessionHistory bindings. Add a content-free thread_route_events store (migration 055) for pre-execution route/experiment metadata: requested provider/model/effort captured at session start, task stratum, experiment/cohort, route event kind, manager/agent ids, and escalation reason. Actual values are derived from measured usage, never copied from the request. usageRouteAttribution composes the base projection with this view and keeps child sessions distinct. T3 carries the metadata only; agent-config remains the routing authority. No prompts, responses, code, or tool bodies are stored.
97b4ceb to
67b4c9c
Compare
|
ROUTE6-1:R2-T3 — RESULT Status: complete (restacked source + migrations + focused tests + docs; draft stacked PR). CI cannot be observed green on the fork — see CI state. Fresh session, not a continuation of the Round-1 T3 session. No merge.
Heads
Rebase / restack method
Conflicts encounteredExactly one conflicted path: Resolution: kept both tests (the #4 M1C interchange fixture test and the #6 durable-history/route-event merge test). Because #6 adds How each #4 correction was preservedThe #6 commit's deletion set is only its intended replacements (verified line-by-line):
Round-2 reconciliation added: Migration id checkOn corrected #4 ( #6 intended additions preserved
Changed paths (all 16)
corrected #4..new #6 diff
Tests (17 files, 312 passed)Added regression ( Typecheck / lint / formatCI state (truthful)Not green. At head Actual agent routeopencode-go / Remaining gaps
Exact next actionRequest independent exact-head review of #6 @ |
|
METRICS-1:M3D dependency update for this history owner T3 #4 (the usage attribution/measurement PR this draft depends on) moved from Two changes are relevant when you eventually rebase onto the new head:
RESULT with before/after receipts: #4 (comment) |
What this is
Narrow follow-on to #4, restacked onto its corrected branch head
daa55eb88966665faa8c1770c35fc3c53a68f6fa(Round-2 R2-T3). It does notchange #4; it adds the minimum durable identity/history and pre-execution
experiment metadata that #4 explicitly deferred, so the ROUTE6-1 coding-model
canary can be measured across resumes, provider-session replacement, model
switches, child sessions, retries/fallbacks, multiple PRs, and manager/agent
assignments.
Draft; not for merge. No endpoint, UI, PostHog, collector, gateway, routing, or
transport change. Upstream
pingdotgg/t3codeis untouched. T3 is not thecanonical model router — agent-config remains policy authority; T3 only carries
and measures metadata.
daa55eb88966665faa8c1770c35fc3c53a68f6fa67b4c9c48e5def2f9bb97547ecb893d794d11d2afeat/durable-model-canary-attribution-20260923Round-2 restack (R2-T3)
Restacked with
git rebase --onto daa55eb88 e4f36af5e. The single #6 commit wasreplayed; only
usageAttributionSources.test.tsconflicted (both sides appended atest to the same
describe), resolved by keeping both tests and extending theM1C interchange fixture expectation with the additive
history/routeEventsfields. Every #4 Round-3 correction is preserved — nothing here reverts the
measurement/invalid/completeness,
dedupeKeyScope, cost-only conflict, snapshotreplacement, warm-cache enrichment, or
readResumeCursorpreference behavior.The files that implement them (
usageTranscripts.ts,usageScanCache.ts,UsageService.ts) are untouched here.Why #4 needed this
#4's own limitation: a native session maps to a thread only through the single current
provider_session_runtime.resume_cursor_json. A resume, fork, or model switch overwrites it and earlier usage becomes unbound. #4 recorded that as the follow-on it did not adopt.Schema decision (migration only where required)
A migration is used because the existing schema cannot truthfully preserve history — the cursor is one row per thread. Two additive, forward-compatible tables (existing databases are safe; existing thread/session behavior is unchanged). Ids 054/055 were confirmed free on the corrected #4 base (its highest existing migration is 053). No collision, no renumbering.
054_provider_session_history— append-only identity,UNIQUE (thread_id, provider_name, native_session_id).ProviderSessionRuntime.upsertappends one row per distinct native session; a repeat only advanceslast_seen_at. The migration backfills the current cursor of an upgraded database. A conflictingonConflict: "ignore"write appends nothing (that cursor was never applied).parent_native_session_idcarries a true sub-agent parent when known.055_thread_route_events— append-only, content-free pre-execution route/experiment metadata. Nullableroute_event_kindkeeps an automatic "requested" record distinct from a declared canary/fallback/review/escalation event.Changed paths
apps/server/src/persistence/Migrations/054_ProviderSessionHistory.tsapps/server/src/persistence/Migrations/055_ThreadRouteEvents.tsapps/server/src/persistence/Migrations.tsapps/server/src/persistence/ProviderSessionRuntime.tsapps/server/src/usage/routeMetadata.tsapps/server/src/usage/usageAttributionSources.tsapps/server/src/usage/usageAttribution.tssessionHistorybinding originapps/server/src/usage/usageRouteAttribution.tsapps/server/src/provider/Services/ProviderSessionDirectory.ts,Layers/ProviderSessionDirectory.tsparentNativeSessionId,requestedRoute,routeEventon the bindingapps/server/src/provider/Layers/ProviderService.tsusageRouteAttribution.test.ts(incl. #4 quality regression),usageAttributionSources.test.ts,ProviderSessionRuntime.history.test.ts,054_ProviderSessionHistory.test.tsdocs/internals/usage-attribution.mdSemantics encoded
projection_thread_pull_requests, reused, not replaced). A readable manager/agent id (ROUTE6-1/T3) is a label, never a substitute for the native session id and never a join key.actualEffortisnullwith qualityunsupported.measurement,identity,prompt,request,recordIdentity,conflict) ornullwhen no usage was measured, so apartial/invalidmeasurement or a legacy identity-erased row is never presented as exact.normal | availability_fallback | canary | independent_review | quality_escalation), and escalation/fallback reason. Unknown is allowed and is the default.shared/attributedrules are unchanged.Validation
Exact per-test receipts and CI state are in the RESULT comment. CI is not
green: the self-hosted
blacksmith-*jobs stay queued on the fork.Model/harness: opencode-go /
deepseek-v4.1-flashvia opencode (see RESULT).