[METRICS-M3] Prove prompt request session and PR usage attribution - #4
nullStack65 wants to merge 7 commits into
Conversation
Public usage summaries aggregate by day/provider/model and the underlying UsageRecord rows differ per provider, so there is no single provider-request unit today. Add a pure projection that re-projects measured usage onto prompt, provider-request, native-session, and pull-request levels using only explicit existing bindings (provider_session_runtime resume cursors and imported transcripts) and existing thread/PR links. Granularity is asserted per source, never inferred: Claude exposes per-request identity, Grok per prompt, Codex only per turn. The projection reports unsupported levels as null instead of dividing a turn. A session linked to several PRs feeds a non-additive shared pool rather than being cloned onto each link, and a known session with no usage is missing (null), never zero. Adds the native request/message/prompt ids to UsageRecord and the v4 scan cache so they stay separate from dedupeKey. Model/harness: opencode-go/deepseek-v4.1-flash via opencode.
github-actions
Bot
added
vouch:trusted
A known session with no measured usage reported equestCount: 0 while its quality was missing, which reads as a zero-request success. Return null unless the level's quality is measured or partial, so absence stays absent. Model/harness: opencode-go/deepseek-v4.1-flash via opencode.
Adds the METRICS-M1 invariant sum(attributed PR totals) + shared + unallocated = distinct measured total, and records that the legacy projection_thread_sessions.provider_session_id columns are not written by the live projector. Model/harness: opencode-go/deepseek-v4.1-flash via opencode.
METRICS-M3 RESULT — prompt / request / session / PR usage attribution proofStatus: complete (source-only proof + focused offline tests + this draft PR). Draft; not for merge. No endpoint, storage, collector, gateway, PostHog, or routing change.
What this proves vs. what still needs approvalProven here (source + fixtures): the native-session → T3-thread → PR join is expressible from persisted state alone; the four levels can be projected with honest per-source granularity; association and allocation can be kept separate; missing is representable as Not proven / needs approval: live qualification of any source; durable per-thread session history (the current resume cursor is single-valued); provider-instance identity at record level; an OpenCode adapter; any storage/transport choice; any cost/invoice basis. Concrete diff
The projection reads only allowlisted metadata: provider, native session id, model, timestamp, token totals, priced cost, native ids, source fingerprint; plus explicit bindings and existing PR links. It never reads prompts, responses, tool output, titles, or paths. Source-capability matrix (source-grounded;
|
| Provider | Session | Prompt | Request | Native unit |
|---|---|---|---|---|
| Claude Code | supported | unsupported | supported | one assistant message = one API response (message.id + requestId) |
| Codex | supported | unsupported | unsupported | token_count delta, one per model turn |
| Grok Build | supported | supported (prompt_id) |
unsupported | turn_completed, one per prompt per model |
- Codex turn-level usage is never divided into requests:
requestCountisnull, qualityunsupported. - Claude has no prompt id, so a prompt spanning several requests stays at request level.
- OpenCode / Antigravity / Cursor expose a native cursor id but no transcript T3 scans → no usage source (
nativeSource: "none"when supplied). Complementary to M1's OpenCode-store slice; this projection is record-agnostic and can consume OpenCode records once an adapter emits them. - CLIProxyAPI in this repo is a quota/limit source only (
auth-files, credits,api-call); it does no token accounting, so it is not an accounting authority and must not be for one. - Correction for M1 §2:
projection_thread_sessions.provider_session_id/provider_thread_idexist in migration005but the live upsert inProjectionThreadSessions.ts:19-53never writes them — they are dead columns. The durable mapping isprovider_session_runtime.resume_cursor_json({threadId}Codex,{resume}Claude,{sessionId}ACP/OpenCode) plusruntime_payload_json.importedTranscripts(providerSessionId+ file identity). Also, an upstream PostHog identifier filter does not exist in code; the exclusion is by convention and omission (product-analytics.md), worth knowing before relying on it.
Example output (synthetic fixture; real module output)
Human-readable:
Usage attribution
Sessions: 5
claude:5a128faa-8253-489e-b935-6c08e8e670c0 [measured/attributed] 285 tokens $0.0600 prompts=unsupported requests=2 threads=t1
claude:99999999-8888-7777-6666-555555555555 [missing/missing] missing prompts=unsupported requests=unsupported threads=t5
claude:aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee [measured/unallocated] 125 tokens $0.0100 prompts=unsupported requests=1 threads=t4
codex:019fbbc1-b12c-7360-a685-28c181f0025f [measured/shared] 170 tokens $0.0300 prompts=unsupported requests=unsupported threads=t3
grok:019fec1a-12f7-72f2-9b1f-7778a00aea3c [measured/attributed] 235 tokens $0.0070 prompts=1 requests=unsupported threads=t2
Pull requests:
github.com/acme/repo#12 attributed=520 tokens shared=0 tokens sessions=claude:5a128faa-8253-489e-b935-6c08e8e670c0,grok:019fec1a-12f7-72f2-9b1f-7778a00aea3c sources=agent,manual
github.com/acme/repo#13 attributed=0 tokens shared=170 tokens sessions=codex:019fbbc1-b12c-7360-a685-28c181f0025f sources=manual
github.com/acme/repo#14 attributed=0 tokens shared=170 tokens sessions=codex:019fbbc1-b12c-7360-a685-28c181f0025f sources=manual
Shared (not additive): 170 tokens
Unallocated: 125 tokens
Coverage:
claude session=supported prompt=unsupported request=supported measured=2 missing=1 unbound=0 ambiguous=0
codex session=supported prompt=unsupported request=unsupported measured=1 missing=0 unbound=0 ambiguous=0
grok session=supported prompt=supported request=unsupported measured=1 missing=0 unbound=0 ambiguous=0
Machine-readable (verbatim module output; verbose tokens bodies kept for exactness):
{
"contractVersion": 1,
"generatedAtMs": 1786100000000,
"sessions": [
{
"provider": "claude",
"sessionId": "5a128faa-8253-489e-b935-6c08e8e670c0",
"models": [
"claude-fable-5"
],
"totals": {
"tokens": {
"uncachedInputTokens": 200,
"cachedInputTokens": 20,
"cacheCreationTokens": 0,
"outputTokens": 65,
"reasoningTokens": 0
},
"totalTokens": 285,
"costUsd": 0.06,
"records": 2
},
"quality": "measured",
"promptQuality": "unsupported",
"requestQuality": "measured",
"promptCount": null,
"requestCount": 2,
"boundThreadIds": [
"t1"
],
"providerInstanceIds": [
"claude-default"
],
"bindingOrigins": [
"runtimeCursor"
],
"allocation": "attributed",
"pullRequestKeys": [
"github.com/acme/repo#12"
],
"stackOnlyPullRequestKeys": []
},
{
"provider": "claude",
"sessionId": "99999999-8888-7777-6666-555555555555",
"models": [],
"totals": null,
"quality": "missing",
"promptQuality": "unsupported",
"requestQuality": "missing",
"promptCount": null,
"requestCount": null,
"boundThreadIds": [
"t5"
],
"providerInstanceIds": [
"claude-default"
],
"bindingOrigins": [
"runtimeCursor"
],
"allocation": "missing",
"pullRequestKeys": [],
"stackOnlyPullRequestKeys": []
},
{
"provider": "claude",
"sessionId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
"models": [
"claude-fable-5"
],
"totals": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 15,
"reasoningTokens": 0
},
"totalTokens": 125,
"costUsd": 0.01,
"records": 1
},
"quality": "measured",
"promptQuality": "unsupported",
"requestQuality": "measured",
"promptCount": null,
"requestCount": 1,
"boundThreadIds": [
"t4"
],
"providerInstanceIds": [
"claude-default"
],
"bindingOrigins": [
"runtimeCursor"
],
"allocation": "unallocated",
"pullRequestKeys": [],
"stackOnlyPullRequestKeys": []
},
{
"provider": "codex",
"sessionId": "019fbbc1-b12c-7360-a685-28c181f0025f",
"models": [
"gpt-5.6-sol"
],
"totals": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 60,
"reasoningTokens": 0
},
"totalTokens": 170,
"costUsd": 0.03,
"records": 1
},
"quality": "measured",
"promptQuality": "unsupported",
"requestQuality": "unsupported",
"promptCount": null,
"requestCount": null,
"boundThreadIds": [
"t3"
],
"providerInstanceIds": [
"codex-default"
],
"bindingOrigins": [
"runtimeCursor"
],
"allocation": "shared",
"pullRequestKeys": [
"github.com/acme/repo#13",
"github.com/acme/repo#14"
],
"stackOnlyPullRequestKeys": []
},
{
"provider": "grok",
"sessionId": "019fec1a-12f7-72f2-9b1f-7778a00aea3c",
"models": [
"grok-4.5",
"grok-fast"
],
"totals": {
"tokens": {
"uncachedInputTokens": 200,
"cachedInputTokens": 20,
"cacheCreationTokens": 0,
"outputTokens": 15,
"reasoningTokens": 0
},
"totalTokens": 235,
"costUsd": 0.007,
"records": 2
},
"quality": "measured",
"promptQuality": "measured",
"requestQuality": "unsupported",
"promptCount": 1,
"requestCount": null,
"boundThreadIds": [
"t2"
],
"providerInstanceIds": [],
"bindingOrigins": [
"runtimeCursor"
],
"allocation": "attributed",
"pullRequestKeys": [
"github.com/acme/repo#12"
],
"stackOnlyPullRequestKeys": []
}
],
"prompts": [
{
"provider": "grok",
"sessionId": "019fec1a-12f7-72f2-9b1f-7778a00aea3c",
"promptId": "p1",
"totals": {
"tokens": {
"uncachedInputTokens": 200,
"cachedInputTokens": 20,
"cacheCreationTokens": 0,
"outputTokens": 15,
"reasoningTokens": 0
},
"totalTokens": 235,
"costUsd": 0.007,
"records": 2
},
"models": [
"grok-4.5",
"grok-fast"
],
"boundThreadIds": [
"t2"
],
"allocation": "attributed"
}
],
"requests": [
{
"provider": "claude",
"sessionId": "5a128faa-8253-489e-b935-6c08e8e670c0",
"providerRequestId": "r1",
"providerMessageId": "m1",
"totals": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 40,
"reasoningTokens": 0
},
"totalTokens": 150,
"costUsd": 0.04,
"records": 1
},
"model": "claude-fable-5",
"boundThreadIds": [
"t1"
],
"allocation": "attributed"
},
{
"provider": "claude",
"sessionId": "5a128faa-8253-489e-b935-6c08e8e670c0",
"providerRequestId": "r2",
"providerMessageId": "m2",
"totals": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 25,
"reasoningTokens": 0
},
"totalTokens": 135,
"costUsd": 0.02,
"records": 1
},
"model": "claude-fable-5",
"boundThreadIds": [
"t1"
],
"allocation": "attributed"
},
{
"provider": "claude",
"sessionId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
"providerRequestId": "r9",
"providerMessageId": "m9",
"totals": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 15,
"reasoningTokens": 0
},
"totalTokens": 125,
"costUsd": 0.01,
"records": 1
},
"model": "claude-fable-5",
"boundThreadIds": [
"t4"
],
"allocation": "unallocated"
}
],
"pullRequests": [
{
"key": "github.com/acme/repo#12",
"host": "github.com",
"repository": "acme/repo",
"number": 12,
"threadIds": [
"t1",
"t2"
],
"linkSources": [
"agent",
"manual"
],
"attributed": {
"tokens": {
"uncachedInputTokens": 400,
"cachedInputTokens": 40,
"cacheCreationTokens": 0,
"outputTokens": 80,
"reasoningTokens": 0
},
"totalTokens": 520,
"costUsd": 0.067,
"records": 4
},
"shared": {
"tokens": {
"uncachedInputTokens": 0,
"cachedInputTokens": 0,
"cacheCreationTokens": 0,
"outputTokens": 0,
"reasoningTokens": 0
},
"totalTokens": 0,
"costUsd": 0,
"records": 0
},
"stackAssociationSessions": [],
"contributingSessions": [
"claude:5a128faa-8253-489e-b935-6c08e8e670c0",
"grok:019fec1a-12f7-72f2-9b1f-7778a00aea3c"
]
},
{
"key": "github.com/acme/repo#13",
"host": "github.com",
"repository": "acme/repo",
"number": 13,
"threadIds": [
"t3"
],
"linkSources": [
"manual"
],
"attributed": {
"tokens": {
"uncachedInputTokens": 0,
"cachedInputTokens": 0,
"cacheCreationTokens": 0,
"outputTokens": 0,
"reasoningTokens": 0
},
"totalTokens": 0,
"costUsd": 0,
"records": 0
},
"shared": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 60,
"reasoningTokens": 0
},
"totalTokens": 170,
"costUsd": 0.03,
"records": 1
},
"stackAssociationSessions": [],
"contributingSessions": [
"codex:019fbbc1-b12c-7360-a685-28c181f0025f"
]
},
{
"key": "github.com/acme/repo#14",
"host": "github.com",
"repository": "acme/repo",
"number": 14,
"threadIds": [
"t3"
],
"linkSources": [
"manual"
],
"attributed": {
"tokens": {
"uncachedInputTokens": 0,
"cachedInputTokens": 0,
"cacheCreationTokens": 0,
"outputTokens": 0,
"reasoningTokens": 0
},
"totalTokens": 0,
"costUsd": 0,
"records": 0
},
"shared": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 60,
"reasoningTokens": 0
},
"totalTokens": 170,
"costUsd": 0.03,
"records": 1
},
"stackAssociationSessions": [],
"contributingSessions": [
"codex:019fbbc1-b12c-7360-a685-28c181f0025f"
]
}
],
"shared": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 60,
"reasoningTokens": 0
},
"totalTokens": 170,
"costUsd": 0.03,
"records": 1
},
"unallocated": {
"tokens": {
"uncachedInputTokens": 100,
"cachedInputTokens": 10,
"cacheCreationTokens": 0,
"outputTokens": 15,
"reasoningTokens": 0
},
"totalTokens": 125,
"costUsd": 0.01,
"records": 1
},
"coverage": [
{
"provider": "claude",
"nativeSource": "transcript",
"liveQualified": false,
"session": "supported",
"prompt": "unsupported",
"request": "supported",
"measuredSessions": 2,
"missingSessions": 1,
"invalidSessions": 0,
"unboundSessions": 0,
"ambiguousSessions": 0,
"recordsWithoutSessionId": 0
},
{
"provider": "codex",
"nativeSource": "transcript",
"liveQualified": false,
"session": "supported",
"prompt": "unsupported",
"request": "unsupported",
"measuredSessions": 1,
"missingSessions": 0,
"invalidSessions": 0,
"unboundSessions": 0,
"ambiguousSessions": 0,
"recordsWithoutSessionId": 0
},
{
"provider": "grok",
"nativeSource": "transcript",
"liveQualified": false,
"session": "supported",
"prompt": "supported",
"request": "unsupported",
"measuredSessions": 1,
"missingSessions": 0,
"invalidSessions": 0,
"unboundSessions": 0,
"ambiguousSessions": 0,
"recordsWithoutSessionId": 0
}
],
"limitations": [
"A native session maps to a T3 thread only through the current resume cursor or imported-transcript metadata; a session switch, fork, or restart that overwrote the cursor leaves earlier usage unbound.",
"Provider-instance identity is not recoverable from a transcript scan, so two instances of one provider cannot be told apart at the record level.",
"Request and prompt counts are reported only where the native source writes those ids; a turn-level source reports `unsupported`, never an inferred count.",
"Some known sessions have no measured usage. They are reported as `missing` with a null total; this is not a zero-cost success.",
"Only the newest native session id per thread is durable. Additive retention must land before historical re-attribution is possible."
]
}Reconciliation holds in the fixture: allocated 520 + shared 170 + unallocated 125 = distinct measured 815, and the duplicate Codex scan is counted once.
Validation (run locally, node 24.21.0)
vp test run apps/server/src/usage/usageAttribution.test.ts apps/server/src/usage/usageTranscripts.test.ts apps/server/src/usage/usageScanCache.test.ts apps/server/src/usage/usageAggregation.test.ts
# 4 files, 69 passed
vp run --filter t3 typecheck # no errors
vp lint <changed files> # clean
git diff --check # clean
Note: the checkout had no node on PATH; ran against ~/AppData/Local/t3-build/node-v24.21.0-win-x64, satisfying engines: node ^24.
CI
- PR Vouch, PR Size, Label PR: passed (on an earlier head; the label/CI runs re-triggered per push).
- CI run for head
db1a3db83: https://github.com/nullStack65/t3code/actions/runs/35846007902 — queued, runner not yet assigned on the fork (no CI result available yet). - CodeRabbit: skipped (draft).
- Local focused tests are green; repo-wide checks were not run (per AGENTS.md).
Coverage and limitations
- Bound native session → thread only through the current resume cursor or imported-transcript metadata. A switch/fork/restart that overwrote the cursor leaves earlier usage
unallocated— demonstrated by test. - Provider-instance identity is not recoverable from a transcript scan; two instances of one provider are indistinguishable at record level.
missingsessions carrynulltotals/counts and never enter a pool;invalidClaude ids are flagged;partialmarks levels where some records lack the id.- Duplicate scans de-dupe by
dedupeKeywith a content-signature fallback for keyless Codex turns; duplicate source fingerprints are surfaced as a limitation. - No percentiles, p50/p95/p99, workload strata, or cost-basis separation yet — that is the M1 §4/§5 report layer, not this projection.
- Enum naming differs from M1's
exact|partial|unavailable|ambiguous: here it ismeasured|partial|missing|invalid|unsupported. Rename is mechanical if M1's nomenclature is adopted.
Schema/privacy decisions requiring Kameron
- Durable session-history schema (highest value): an additive record of
(threadId, provider, providerInstanceId, nativeSessionId, firstSeenAt, lastSeenAt, origin)written alongside the resume-cursor upsert, so re-attribution survives switches/forks. Not adopted; the current single-cursor row is the binding. - Provider-instance tagging at scan time (tag each scanned file with the instance that produced it) to disambiguate multi-instance providers.
- Enum/nomenclature alignment with M1 §2 (mechanical).
- Storage/transport for the report remains M1's D2/D3 decision; this PR deliberately chooses none.
- OpenCode adapter ownership (M1 §9 M1b) — this projection can consume it but does not build it.
No privacy expansion: no raw content, no new PostHog field, no conversation identifier added anywhere.
Recommendation
SMALL EXTENSION WORTH LANDING — as a scoped, source-only slice. The projection, the id fields, and the capability/quality contract are small, pure, and tested; they convert "measure usage" from a day/model bucket into a level-honest join without touching storage or telemetry.
If the owner prefers zero schema/cache movement now, the cheaper fallback is DEFER the request/prompt id extension: keep the projection but run it over session + PR levels only (Claude/Codex/Grok all remain unsupported at request/prompt), which leaves usageTranscripts.ts and usageScanCache.ts untouched. That trades the request-level proof for a smaller diff.
Next step
M1/M3 owner review of (1) the enum alignment and (2) the additive durable-session-history schema. On approval, the follow-up is the M1 §9 M1b-style offline exporter (OpenCode store + T3 projection DB) reusing this projection; until then the live sources stay unqualified and unactivated.
Model/harness: opencode-go/deepseek-v4.1-flash via opencode.
|
START METRICS-M3R — bounded T3 attribution repair.
|
|
ROUTE6-1 linkage: this attribution proof is a dependency of nullStack65/closura-agent-config#238. The T3 lane in pingdotgg#238 should build on this PR rather than recreate attribution. The narrow follow-on is durable session/history plus pre-execution experiment metadata sufficient to preserve manager/agent/model/PR attribution across resumes, model switches, and child sessions. T3 must not become the canonical model router. Please post future durable RESULT/PR links back to agent-config#238. |
A v3 scan cache is read instead of discarded, so measured records from deleted transcripts survive; v3 rows decode with native ids and presence explicitly unavailable and extant files are cold re-parsed to enrich them. Parsers record whether tokens were actually observed, and the occurrence-aware dedupe key moves to a shared seam reused by the scan.
Preserve orphan usage in an explicit bucket and reconcile attributed + shared + unallocated + orphan against the deduplicated input. Replace content-equality dedupe with a provider-namespaced declared key plus occurrence-aware scan identity, keeping unkeyed records uncertain and surfacing conflicts and snapshot replacement. Split identity, measurement, level support, and allocation; seed coverage from declared sources; retain per-model and cost provenance. Add a read-only extraction seam from provider_session_runtime and projection_thread_pull_requests.
|
CHECKPOINT METRICS-M3R — repair pushed.
|
|
RESULT METRICS-M3R — bounded T3 attribution repair. Status: complete (source + focused tests + docs; draft PR). CI cannot be observed green on the fork (see Checks).
Touch set
No endpoint/UI, PostHog, storage/history schema, collector, gateway/routing, install, merge, or activation change. Upstream Per-finding dispositions
Reconciliation evidence
ChecksLocal (worktree, off reviewed head): CI (run Capability / coverage limits
Metadata interface M1R can consume (read-only, no payload)Ready for independent reviewThe proof is source-grounded and fixture-tested only. Requested review focus: the identity model (declared vs occurrence vs unkeyed), the reconciliation identity, and the v3 read/enrich path. |
|
ROUTE6-1:T3 follow-on — durable model-canary attribution, stacked on this PR's head
This PR's branch and reviewed head are unchanged. Do not merge. |
Classify usage fields by validity and provider completeness instead of property presence: an invalid value is invalid, a valid known subset is partial, and a nonzero total never implies a complete measurement. Make dedupe-key scope explicit. A source-local key is qualified by its native session; a global key reused under another session is incompatible ownership surfaced as a conflict. Cost and provenance are part of the observation, so a repriced record conflicts rather than silently deduping. Require the current identity format for a warm scan-cache hit so an unchanged legacy transcript is cold re-parsed once to enrich it, keeping deleted history and read-failure fallbacks. Carry the legacy identity-erased marker through to the projection so an erased native id is unavailable, not missing.
|
M1C BRIDGE FIXTURE — Artifact for METRICS-1:M1C (the report's
It is generated from persisted-row shapes only — no runtime payload is ever returned. The input rows are the allowlisted {
"cutoffMs": 1786100000000,
"bindings": [
{
"threadId": "import:claude-original:original-session",
"provider": "claude",
"providerInstanceId": "claude-default",
"nativeSessionId": "5a128faa-8253-489e-b935-6c08e8e670c0",
"origin": "runtimeCursor"
},
{
"threadId": "import:claude-original:original-session",
"provider": "claude",
"providerInstanceId": "claude-original",
"nativeSessionId": "original-session",
"origin": "importedTranscript"
},
{
"threadId": "thread-codex",
"provider": "codex",
"providerInstanceId": "codex-default",
"nativeSessionId": "019fbbc1-b12c-7360-a685-28c181f0025f",
"origin": "runtimeCursor"
}
],
"nativeSessions": [
{
"threadId": "thread-opencode",
"providerName": "opencode",
"adapterKey": "opencode",
"providerInstanceId": "opencode-default",
"nativeSessionId": "ses_opencode_1",
"origin": "runtimeCursor",
"usageProvider": null
},
{
"threadId": "import:claude-original:original-session",
"providerName": "claudeAgent",
"adapterKey": "claudeAgent",
"providerInstanceId": "claude-default",
"nativeSessionId": "5a128faa-8253-489e-b935-6c08e8e670c0",
"origin": "runtimeCursor",
"usageProvider": "claude"
},
{
"threadId": "import:claude-original:original-session",
"providerName": "claudeAgent",
"adapterKey": "claudeAgent",
"providerInstanceId": "claude-original",
"nativeSessionId": "original-session",
"origin": "importedTranscript",
"usageProvider": "claude"
},
{
"threadId": "thread-codex",
"providerName": "codex",
"adapterKey": "codex",
"providerInstanceId": "codex-default",
"nativeSessionId": "019fbbc1-b12c-7360-a685-28c181f0025f",
"origin": "runtimeCursor",
"usageProvider": "codex"
}
],
"links": [
{
"threadId": "thread-opencode",
"host": "github.com",
"repository": "acme/repo",
"number": 12,
"source": "manual",
"linkedAt": "2026-09-01T00:00:00.000Z",
"url": "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/acme/repo/pull/12"
},
{
"threadId": "thread-codex",
"host": "github.com",
"repository": "acme/repo",
"number": 12,
"source": "agent",
"linkedAt": "2026-09-01T00:00:00.000Z",
"url": "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/acme/repo/pull/12"
},
{
"threadId": "thread-codex",
"host": "github.com",
"repository": "acme/repo",
"number": 13,
"source": "stack",
"linkedAt": "2026-09-01T00:00:00.000Z",
"url": "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/acme/repo/pull/12"
}
],
"diagnostics": {
"bindings": {
"runtimeRows": 3,
"runtimeCursorBindings": 3,
"importedTranscriptBindings": 1,
"absentIdentityRows": 0,
"malformedResumeCursors": 0,
"malformedRuntimePayloads": 0,
"skippedImportedTranscripts": 0,
"unsupportedProviderBindings": 1,
"overwrittenThreads": 1,
"ambiguousSessionIds": 0
},
"links": {
"rows": 3,
"links": 3,
"dismissed": 0,
"malformed": 0
}
}
}Allowlisted field semantics (one narrow interchange; no second report/reader is added):
Retained: |
|
RESULT METRICS-1:M3C — finish T3 measurement and cache correctness Status: complete (source + focused tests + docs; draft PR). Agent label METRICS-1:M3C. Session reference: unknown (openCode/T3 harness exposes no native session URL/id to this agent; not reused from METRICS-M3/M3R).
Touch set (11 files, +1070/-72)
Dispositions of the three material residuals
Exact commands / results (local, isolated worktree)
Reproduced-before-fixing: with Reconciliation evidence
M1C bridge fixturePosted as a separate #4 comment (#4 (comment)): the exact CI (honest, refreshed at corrected head)Run for Remaining limits and next action
Backlinks: https://github.com/nullStack65/closura-agent-config/pull/231 · https://github.com/nullStack65/closura-agent-config/pull/238 · follow-on owner notified on #6. Model/harness: opencode-go/deepseek-v4.1-flash via opencode. Agent label METRICS-1:M3C; session reference unknown. |
|
START METRICS-1:M3D — preserve zero/invalid usage and cache quality
|
|
METRICS-1:M1D fixture note (from closura-agent-config pingdotgg#231). The M1D core interchange fixture is now generated by the real The report side consumes |
Retain eligible zero-total observations instead of dropping them before classification. Codex no longer returns null for a zero subtotal, and Grok no longer skips zero-total aggregate or per-model rows: a complete measured zero, a known-zero subset, and an all-invalid payload all reach the projection with their quality, while a container with no recognised token field is still no-usage. Grok per-model cost accounting now reduces the aggregate for every emitted ticked row, so an emitted zero-token row cannot double count. Treat missing completeness metadata as unasserted. The predecessor v4 writer emitted 15-field rows without it, and the decoder defaulted a non-legacy observed row to complete, silently promoting an unknown measurement. Missing completeness now decodes as partial, and the entry is marked qualityMetadata: "predecessor" so warm-cache acceptance and resume both require the current quality format. An extant file is cold re-parsed once; deleted and unreadable history is retained conservatively. Native-id availability is a separate axis from numeric freshness. Regressions cover Codex/Grok complete zero, partial zero, invalid, and valid-plus-invalid distinct events, the parser-to-projection path, a pinned predecessor fixture generated by the actual predecessor writer, and the service-seam refresh/restart/deleted/failed-read paths.
|
RESULT METRICS-1:M3D — preserve zero/invalid usage and cache quality Status: complete (source + focused tests + docs; draft PR updated in place). Agent label METRICS-1:M3D, round 4. Session reference: unknown (harness exposes no native session URL/id; fresh session, not continued from an earlier agent session).
Touch set (8 files, +829/−72): REPAIR 1 — zero-total filters erase quality: FIXED
Independent execution of eligible source-shaped events now:
Direct parser + parser→projection regressions added ( REPAIR 2 — cache measurement freshness: FIXEDSmallest parser-format/freshness repair, no version bump, no history discarded. Supported-format policy (explicit):
The v3 identity-enrichment path is unchanged and still passes its service-level regression (one-time refresh, warm second scan byte-identical, restart, deleted source). Before/after evidence. A fixture was generated by the actual predecessor writer ( That is the silent promotion reproduced (the row whose source was Service-seam receipts (
Exact commands / environment (local, honest)Ran in WSL Ubuntu 24.04, Node The previous result pinned CI (exact state for
|
This branch has not been deployed
1 participant
What this is
METRICS-M3R repair slice, continued through METRICS-1:M3D. T3's public usage summaries
aggregate by
(day, hour, provider, model), and the underlyingUsageRecordrows differper provider, so there is no universal provider-request unit. This PR proves the cheapest
path from the measurements that already exist to four reporting levels — prompt,
provider request, native session, and pull request — without inventing a storage
backend, endpoint, or identifier in PostHog, and repairs the reproduced attribution
failures plus the persisted-binding proof.
Draft; not for merge. No endpoint, UI, PostHog, storage schema, collector, gateway,
routing, or installation change. Upstream
pingdotgg/t3codeis untouched.main):bcc1a58b19a9d610a4f08fed191a364767bc65b3daa55eb88966665faa8c1770c35fc3c53a68f6faaf70495442678a06e007d13bba477f123170c0f5Findings repaired
an explicit
orphanbucket, coverage seeds from declared sources and recordsbefore any session exists, and the reconciliation identity holds against the
deduplicated input, not a pre-filtered session list:
sum(PR.attributed) + shared + unallocated + orphan === measured.dedupeKeyis the scan/deliveryidentity with an explicit
dedupeKeyScope:globalkeys are provider-namespaced andcollapse copies at another path;
source-localkeys (the Codex occurrence seam) arequalified by the canonical native session, so equal local keys in two sessions stay
distinct. A global key reused under a second session is incompatible ownership
surfaced as a conflict. Cost and provenance are part of the observation, so a
repriced record conflicts instead of silently deduping. Unkeyed records are counted
and surfaced as
uncertain, not merged. Asnapshotrecord replaces instead ofadding.
once. A warm-cache hit now requires both the current identity/measurement format
and the current numeric-quality metadata, so an unchanged extant v3 transcript — or
an unchanged v4 transcript whose rows predate the completeness fields — is cold
re-parsed once to fill in ids and quality without duplicate totals. Deleted-transcript
history survives, a read failure keeps the retained fallback rows, and a deleted file
is never re-parsed. A legacy nonzero row keeps its
identityAvailable: falsemarkerthrough to the projection, so an erased native id is
unavailable, nevermissingand never recovered from token magnitude. A predecessor row's missing completeness
decodes as
partial, nevercomplete.one. Parsers classify token fields by actual validity and provider completeness,
not property presence: an invalid value (
null, a string, a negative number) isinvalid; a valid known subset (Claudeinput_tokenswithoutoutput_tokens) ispartial; an explicit valid zero ismeasured; a legacy erased row isunavailable. Codex and Grok now retain an eligible event whose total is zero(complete zero, known-zero subset, or all-invalid payload), so its quality reaches
the projection instead of vanishing at a parser gate; only a container with no
recognised token field is no-usage. A nonzero total never implies a complete
measurement. Identity validity, measurement completeness, level support, source
coverage, and allocation certainty are separate axes.
usageAttributionSources.tsis a read-only extractionseam from actually written
provider_session_runtime.resume_cursor_json/runtime_payload_json.importedTranscriptsandprojection_thread_pull_requestsintothe allowlisted binding/link snapshot, with diagnostics for absent, malformed,
overwritten, and ambiguous history. It never returns a runtime payload.
What changed
apps/server/src/usage/usageAttribution.ts— projection rewrite: orphan bucket,provider-namespaced + occurrence-aware identity, conflict/snapshot handling, split
identity/measurement/level/source/allocation axes, per-model contributions with cost
provenance, association basis.
apps/server/src/usage/usageAttributionSources.ts(new) — read-only extraction frompersisted runtime and PR-link rows.
apps/server/src/usage/usageTranscripts.ts—UsageMeasurement(+invalid),UsageMeasurementCompleteness,DedupeKeyScope, the shared occurrence seam, fieldvalidity/completeness classification, Claude
usage: {}detection, and (round 4)zero-total retention for Codex/Grok plus emitted-tick cost accounting.
apps/server/src/usage/usageScanCache.ts— v3 read + legacy marker, measurement /completeness / key-scope serialization, legacy identity-erased marker, and (round 4)
qualityMetadata: "declared" | "predecessor"with missing completeness decoding aspartial.apps/server/src/usage/UsageService.ts— reuse the seam, warm-cache acceptancerequires the current identity and quality format, no incremental resume of a legacy
or predecessor entry, count
usage: {}and invalid containers asmalformedRecords.usageAttribution.test.ts,usageAttributionSources.test.ts,usageScanCache.test.ts,usageTranscripts.test.ts,UsageService.test.ts.docs/internals/usage-attribution.md.Source capability (source-grounded; live qualification is false for all rows)
message.id+requestId)token_countdelta, one per model turnprompt_id)turn_completed, one per prompt per modelUsageProviderKindisclaude | codex | grok. OpenCode, Antigravity, and Cursor have anative cursor id but no transcript T3 scans, so they have no usage source here; they are
exposed only as
ExtractedNativeSessionwithusageProvider: null. No OpenCode parseris added and the public union is not widened. Turn-only data remains turn-only.
Correctness rules encoded
non-additive
sharedpool, never cloned onto each link.stacklinks areassociation-only;
stack-dismissedtombstones are ignored.generatedAtMs(association.basis: "links-at-read-time").linkedAtdoes not gate allocation, so pre-link implementation work is included;a changed link rewrites the recomputed view, and historical allocation as of a past
instant is unavailable without temporal evidence.
reason to drop an observation.
quality metadata is never silently promoted to
complete.costSourceispreserved or reported
unknown.Validation (local receipts; CI is separate and reported in the RESULT)
Round 3 (METRICS-1:M3C)
The three material residuals from the independent review are repaired: numeric
validity/completeness, explicit identity scope with cost-only conflicts, and warm legacy
cache enrichment (with a service-level regression proving enrichment happens once,
totals stay stable, and request-ID availability is truthful). The M1C interchange fixture
is generated by the real extractor from persisted-row shapes only and posted as a #4
comment. Corrected head
daa55eb88966665faa8c1770c35fc3c53a68f6fa; #6 must eventuallyrebase onto it.
Round 4 (METRICS-1:M3D)
Two bounded repairs with before/after evidence:
nulland Grokdropped aggregate/per-model rows when the normalized total was zero, before numeric
classification, so an all-invalid payload never reached
UsageService'smalformedRecordsand a measured zero never reached session quality. Now an eligibleevent is emitted whenever its usage container has a recognised token field, with its
real
measurementand completeness; only a no-field container is no-usage. Grokper-model cost accounting reduces the aggregate for every emitted ticked row, so an
emitted zero-token row cannot double count, while a row with no token field does not
silently swallow its ticks.
predecessor head
e4f36af5ewrote 15-field v4 rows with no completeness metadata,and the current decoder defaulted a non-legacy observed row to
complete, promotingan unknown measurement and letting
identity: "declared"bypass the correctedparser on an unchanged file. Supported-format policy is now explicit: missing
completeness decodes as
partial(nevercomplete), the entry isqualityMetadata: "predecessor", and warm acceptance + incremental resume bothrequire the current quality format. An extant file is cold re-parsed once and then
warms; deleted/unreadable history is retained conservatively; v3 identity enrichment
is unchanged. No database or session-history migration.
Before/after receipt on the pinned predecessor fixture (generated by the actual
predecessor writer at
e4f36af5e, verbatim 15-field rows): the pre-fix decoder atdaa55eb88readscompleteness=["complete","complete","complete"]; the round-4 decoderreads
["partial","partial","partial"]. Service-seam regressions cover one-timerefresh, warm/cold parity, restart, deleted source, and failed-read fallback.
The M1D bridge contract is unaffected:
usageAttributionSources.tsand the postedextractor snapshot did not change, so the M1C interchange fixture remains byte-stable.
Not in scope / needs Kameron
No endpoint, no persistence choice, no PostHog change, no live collection. Durable
per-thread session history (an additive identity record instead of the single current
resume cursor) and pre-execution experiment metadata are the follow-on that would widen
coverage; specified but not adopted. Provider-instance identity is not recoverable from
a transcript scan. No fixture-tested coverage here is claimed as live-qualified.
Model/harness: opencode-go/deepseek-v4.1-flash via opencode.