Skip to content

fix(custody): manifest shape and apiKey on refusing loader returns - #231

Merged
ualtinok merged 3 commits into
cortexkit:mainfrom
iceteaSA:fix/custody-cutover
Sep 17, 2026
Merged

ualtinok merged 3 commits into
cortexkit:mainfrom
iceteaSA:fix/custody-cutover

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Two production defects found during a live global-custody cutover on this machine. Both were only reachable once main and an enabled fallback were actually tombstoned, which is why the test suite was green throughout.

1. Handle-manifest block written without shape

writeCustodyHandleManifestEntryLocked created our anthropic block with provider / serve / accounts but no shape. The co-tenant Claustrum plugin's parseHandleFile validates every provider block (item.shape !== "api" && item.shape !== "oauth" → throw) before it filters blocks by serve ownership, so one malformed block rejects the whole shared file.

Effect: after we enrolled an account, unrelated providers in the same manifest (MiniMax, DeepSeek, Synthetic) started failing with provider 3 has invalid shape; tombstone ownership cannot be proven. Our own reader was unaffected, so nothing on our side reported a problem.

Fix: emit shape: 'oauth' on new blocks, and repair an existing block that lacks it rather than taking the unchanged fast path.

Worth noting for anyone auditing the co-tenant contract: within a provider P's block the credential id must satisfy credential_id.split(':')[1] === P. Segment 1 is an open set (oauth:, chatgpt:, apikey:, antigravity: are all live), so do not constrain it.

2. Refusing auth.loader returns omitted apiKey

Both custody-mismatch branches in auth.loader returned { fetch } only:

custodyStartupMismatchVerdict = error.verdict
return { fetch: async () => { throw error } }

@ai-sdk/anthropic resolves credentials in getBetasFromHeaders/doStream before it ever calls the custom fetch, so an absent apiKey threw AI_LoadAPIKeyError first. The user-visible error was:

Anthropic API key is missing. Pass it using the 'apiKey' parameter or the ANTHROPIC_API_KEY environment variable.

…for a machine with no API key configured and no API key involved — while the real diagnosis (RESUME_TAKEOVER / FAIL_CLOSED, i.e. an incomplete takeover) never surfaced. The serving branch already returned apiKey: '' and the SDK accepts an empty string, so the refusal branches were simply inconsistent with their sibling.

Why the existing test did not catch it: the fixture called result.fetch directly and asserted the custody error. That path is real, but it is not the path production takes — the SDK never reached the callback. A test can exercise a refusal thoroughly and still prove nothing about the code path the SDK actually walks.

The new regression drives real @ai-sdk/anthropic@3.0.111 doStream against the real auth.loader output (hermetic: temp storage/manifest, stub connector, throwing global fetch, no network, no credentials). Reverting the production change reddens it with the exact LoadAPIKeyError; with the fix, the custody error propagates as intended. @ai-sdk/anthropic is added as a pinned devDependency only — no production import.

The second refusal branch is unreachable through normal loader flow (reconcileCustodyStartup rejects those dimension combinations earlier); it is patched defensively for consistency, and the report says so rather than claiming coverage.

Verification

  • packages/core 199 pass · packages/opencode 1879 pass · packages/pi 114 pass · packages/e2e-tests 36 pass — all 0 fail
  • root typecheck, format:check, biome check . clean
  • red-first proof on both fixes; per-hunk, not whole-diff
  • independent cross-family review: APPROVE, 0 must / 1 should (test env restoration, fixed in this branch)

Live, on the machine where the outage happened — both Anthropic routes now serve under custody with the local slots holding only tombstones:

route=fallback_0  200  direct
route=main        200  direct

A 200 on route=main is only possible from the vault, since that auth slot holds an empty-access tombstone.

Not claimed here: the exact background request count during those probes (Prime/CacheKeep were enabled and their sends are not captured in dump artifacts), and the manifest shape repair does not recover an already-running host — a plugin that parsed a malformed manifest at config time captures the refusal in a closure and needs a restart.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Fixes three production defects found during a live global-custody cutover: handle-manifest blocks were missing shape, refusing auth.loader branches omitted apiKey, and vault-served fallback accounts showed no quota.

Bug Fixes

  • The co-tenant manifest parser validates every provider block's shape before filtering by serve, so a missing shape rejected the whole shared file; new blocks emit shape: 'oauth' and existing blocks are repaired.
  • The AI SDK resolves credentials before calling the custom fetch, so the fetch-only refusal raised AI_LoadAPIKeyError; the refusal branches now return apiKey: '' like the serving branch.
  • Sidebar quota was gated on local account.access, which a vault-served tombstone never has; it now also accepts vaultServed while the lineage-bound cache read still guards stale snapshots.
  • The regression test drives real @ai-sdk/anthropic doStream against the loader output; @ai-sdk/anthropic is a pinned devDependency only.

Written for commit c572508. Summary will update on new commits.

Review in cubic

The co-tenant Claustrum parser validates every provider block's shape before filtering foreign serve ownership, so an anthropic block without shape rejected the whole shared file and broke unrelated providers. Emit shape on new blocks and repair an existing block that lacks it.
The AI SDK resolves credentials before invoking the plugin's custom fetch, so a fetch-only return raised a missing-API-key error instead of the intended custody mismatch. Existing tests called the loader's fetch directly and never exercised that path; the new regression drives the real SDK.
@socket-security

socket-security Bot commented Sep 17, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​ai-sdk/​anthropic@​3.0.111991008898100

View full report

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 6 files

Confidence score: 5/5

  • packages/opencode/src/tests/custody-loader-sdk.test.ts does not assert the returned apiKey, so @ai-sdk/anthropic could fall back to ANTHROPIC_API_KEY and mask a regression in loader handling; add an explicit assertion for the returned key.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/src/tests/custody-loader-sdk.test.ts">

<violation number="1" location="packages/opencode/src/tests/custody-loader-sdk.test.ts:131">
P3: The test never asserts the returned `apiKey`, even though this PR's second fix is specifically about refusing loader returns correctly providing `apiKey`. `@ai-sdk/anthropic` falls back to the `ANTHROPIC_API_KEY` environment variable when `apiKey` is missing from the settings, so if a later change regresses the loader to omit `apiKey` again (the exact bug this PR fixes), this test can still pass on any machine where that env var is set: `createAnthropic({ apiKey: undefined })` resolves the env key, reaches the wrapped `loaded.fetch`, and the `rejects.toMatchObject`/`fetchCalls === 1` assertions stay green — while production without the env var surfaces the original missing-API-key error instead of the custody error. Assert directly on the loader contract (e.g. `loaded.apiKey` is defined/equals the value the loader currently returns) immediately after the loader call, before constructing the SDK model.</violation>
</file>
Architecture diagram
sequenceDiagram
    participant SDK as @ai-sdk/anthropic
    participant Loader as auth.loader
    participant Plugin as AnthropicAuthPlugin
    participant Manifest as Handle Manifest
    participant Connector as Claustrum Connector
    participant Vault as Credential Vault

    Note over Plugin,Manifest: Manifest Write Flow (Fix 1)
    
    Plugin->>Manifest: writeCustodyHandleManifestEntryLocked()
    alt No existing block
        Manifest->>Manifest: Create provider block with shape: 'oauth'
    else Existing block missing shape
        Manifest->>Manifest: Repair block, add shape: 'oauth'
    else Block unchanged (has shape)
        Manifest->>Manifest: Return { status: 'unchanged' }
    end
    
    Note over Manifest,Connector: Co-tenant Parser Validation
    Manifest->>Manifest: All provider blocks validated for shape
    alt Invalid shape
        Manifest-->>Plugin: Reject file (ownership unprovable)
    else Valid shape
        Manifest-->>Plugin: Accept manifest
    end

    Note over SDK,Vault: Auth Loader Flow (Fix 2)
    
    SDK->>Loader: Request credentials for stream
    Loader->>Plugin: Resolve custody state
    Plugin->>Manifest: Check provider block shape
    Manifest-->>Plugin: Shape valid
    
    alt Custody mismatch (RESUME_TAKEOVER)
        Plugin->>Plugin: Get custody verdict
        Plugin-->>Loader: Return { apiKey: '', fetch: throw custody error }
        Loader-->>SDK: Expose refusal fetch
        SDK->>SDK: Resolve API key (empty string accepted)
        SDK->>SDK: Call fetch()
        SDK->>Loader: Throw CustodyStateMismatchError
        Loader-->>SDK: Propagate error (code: custody_state_mismatch)
        SDK-->>SDK: Stream fails with real custody error
    end
    
    alt Custody mismatch (takeover-incomplete)
        Plugin-->>Loader: Return { apiKey: '', fetch: refusal }
        Loader-->>SDK: Expose refusal fetch
    end
    
    Note over Plugin,Vault: Happy Path - Valid Custody
    Plugin->>Connector: credential.get(handle)
    Connector->>Vault: Request credential
    Vault-->>Connector: Return vault credential
    Connector-->>Plugin: Valid credential
    Plugin-->>Loader: Return { apiKey, fetch: claustrumFetch }
    Loader-->>SDK: Normal auth flow
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

const loaded = (await plugin.auth.loader(
() => Promise.resolve(custodyTombstoneOAuth('anthropic') as never),
{ models: {} },
)) as { apiKey?: string; fetch: typeof fetch }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The test never asserts the returned apiKey, even though this PR's second fix is specifically about refusing loader returns correctly providing apiKey. @ai-sdk/anthropic falls back to the ANTHROPIC_API_KEY environment variable when apiKey is missing from the settings, so if a later change regresses the loader to omit apiKey again (the exact bug this PR fixes), this test can still pass on any machine where that env var is set: createAnthropic({ apiKey: undefined }) resolves the env key, reaches the wrapped loaded.fetch, and the rejects.toMatchObject/fetchCalls === 1 assertions stay green — while production without the env var surfaces the original missing-API-key error instead of the custody error. Assert directly on the loader contract (e.g. loaded.apiKey is defined/equals the value the loader currently returns) immediately after the loader call, before constructing the SDK model.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/tests/custody-loader-sdk.test.ts, line 131:

<comment>The test never asserts the returned `apiKey`, even though this PR's second fix is specifically about refusing loader returns correctly providing `apiKey`. `@ai-sdk/anthropic` falls back to the `ANTHROPIC_API_KEY` environment variable when `apiKey` is missing from the settings, so if a later change regresses the loader to omit `apiKey` again (the exact bug this PR fixes), this test can still pass on any machine where that env var is set: `createAnthropic({ apiKey: undefined })` resolves the env key, reaches the wrapped `loaded.fetch`, and the `rejects.toMatchObject`/`fetchCalls === 1` assertions stay green — while production without the env var surfaces the original missing-API-key error instead of the custody error. Assert directly on the loader contract (e.g. `loaded.apiKey` is defined/equals the value the loader currently returns) immediately after the loader call, before constructing the SDK model.</comment>

<file context>
@@ -0,0 +1,171 @@
+      const loaded = (await plugin.auth.loader(
+        () => Promise.resolve(custodyTombstoneOAuth('anthropic') as never),
+        { models: {} },
+      )) as { apiKey?: string; fetch: typeof fetch }
+      let fetchCalls = 0
+      const model = createAnthropic({
</file context>

Sidebar quota was gated on truthy local account.access, which a vault-served tombstone never has, so an enabled fallback rendered no quota while polling it normally. The lineage-bound cache read still guards the stale-snapshot case the gate existed for.
@ualtinok
ualtinok merged commit 3802bb6 into cortexkit:main Sep 17, 2026
5 checks passed
rustybret pushed a commit to rustybret/anthropic-auth that referenced this pull request Sep 17, 2026
3802bb6 Merge PR cortexkit#231: fix manifest shape, apiKey on refusing loader returns, and vault-served fallback quota
81cd538 fix(opencode): use storage refresh window for startup warm minTtlMs
c572508 fix(custody): show quota for vault-served fallback accounts
2d94fd4 Merge PR cortexkit#231: fix manifest shape and apiKey on refusing loader returns
2013161 fix(custody): resolve connection path on macOS, show dialog errors, and fix boot quota
c397d9a fix(custody): keep apiKey on refusing auth.loader returns
47839a1 fix(custody): write and repair shape on our handle-manifest block

# Conflicts:
#	bun.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants