Skip to content

perf(transcription): replace the openai SDK with an in-house client - #366

Merged
chhoumann merged 11 commits into
masterfrom
refactor/drop-openai-sdk
Oct 11, 2026
Merged

chhoumann merged 11 commits into
masterfrom
refactor/drop-openai-sdk

Conversation

@chhoumann

@chhoumann chhoumann commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

PodNotes shipped the whole openai SDK, about half of main.js, to make one kind of request. It now uses a small in-house client, so the plugin is 55% smaller and transcribes as before.

  • createTranscription posts the same multipart request to /v1/audio/transcriptions through fetch, with the caller's abort signal and the SDK's 10-minute timeout. Whisper and OpenAI diarization call it. Error text follows the SDK's rules, so failure notices read the same.
  • It sends once per attempt. The SDK retried inside PodNotes' own three attempts, so a persistent 429, 5xx or network failure cost up to 9 uploads per chunk. It now costs 3. Between attempts, both retry loops wait the delay the server asks for (retry-after-ms or Retry-After, up to 60 s, as openai-node did) and otherwise 1 s, then 2 s.
  • It keeps fetch, not requestUrl, because requestUrl cannot abort an upload in flight, and a cancelled chunk would keep uploading and be billed. One scoped lint exception covers that call.
  • The openai package and the credential cache that held its client are removed.
  • Vite lists its development|production resolve condition again, so Svelte's dev flag is fixed at build time instead of read from process.env in Obsidian. Dev builds keep the flag off, as before, unless Vite itself runs with NODE_ENV=development.

Not included: Deepgram cancellation (still requestUrl) and the doubled "Transcription failed:" prefix on Whisper notices.

Compatibility: removing the SDK also drops its implicit environment overrides (OPENAI_BASE_URL, OPENAI_ORG_ID, OPENAI_PROJECT_ID, OPENAI_CUSTOM_HEADERS, OPENAI_LOG), none of which is a PodNotes setting, and its accept and x-stainless-* telemetry headers. The client expects a JSON response, which both call sites request; it does not support response_format: text.

Verification

  • npm ci, then npm run lint, format:check, typecheck, build, test (1201 tests) and docs:build pass on the merge with master.
  • In an isolated Obsidian 1.13.7 on Linux, with window.fetch replaced by a recorder, Transcribe on a local mp3 sends the same request and shows the same notice as the master build in 7 scenarios, including unload mid-upload. A 429 with Retry-After: 4 recovers on the second upload, as on master (sends at 0 and about 4,000 ms, note written). The comments have the tables and screenshots.
  • There was no live OpenAI call, because this machine has no API key.
  • main.js: 677,800 → 302,862 bytes raw, 177,286 → 95,570 bytes gzip.

Release impact

None for settings, storage, the API or URIs. Every OpenAI transcription request now goes through new code. Persistent failures give up after 3 uploads instead of up to 9.

Note

Replace OpenAI SDK with in-house HTTP client for transcription and diarization

  • Removes the openai package and adds createTranscription in openaiTranscription.ts, a raw fetch helper that builds multipart requests, sends the bearer credential, and has a ten-minute timeout plus caller-abort propagation
  • Retries now honor server-provided Retry-After headers (capped at one minute); invalid or excessive values fall back to the existing incremental delay
  • Each transcription run reads the current OpenAI credential instead of reusing a cached SDK client key, so all credential-cache invalidation code in PodNotes and PodNotesSettingsTab is removed
  • diarizeWithOpenAI now takes an API key instead of a client factory and posts through the shared helper; Whisper responses without a text field are treated as failed attempts
  • Updates all tests to stub global fetch, with new suites covering multipart encoding, error messages, retry timing, timeout, and cancellation
  • Behavioral Change: TranscriptionService.getApiKey is now synchronous; disposal no longer clears any SDK client cache. Also adds development|production to resolve.conditions in vite.config.ts

Macroscope summarized 4bc910c.

Replace the openai SDK with createTranscription, one multipart POST to
/v1/audio/transcriptions through fetch with the caller's abort signal and
the SDK's 10-minute timeout. It sends exactly once, so PodNotes' three
attempts no longer stack on the SDK's own two retries (up to nine sends
per chunk before). Error messages match the SDK's for OpenAI error bodies,
plain-text and empty bodies, network failures and timeouts.

TranscriptionService and the OpenAI diarization provider now pass the API
key instead of a cached SDK client. The credential-cache hook in main.ts
is left empty here and removed with its callers in the next commit.
The cache only held the SDK client so a changed OpenAI key reference could
evict it. TranscriptionService now reads the key per run, so the settings
tab no longer needs to invalidate anything on import or key selection.
Setting resolve.conditions replaces Vite's defaults, which include the
"development|production" condition. Without it esm-env fell back to
reading process.env.NODE_ENV at runtime, so Svelte's dev-only error
branches shipped in the production main.js and dev builds depended on the
shell that launched Obsidian.

Production main.js: 313,095 -> 303,042 bytes raw, 98,849 -> 95,630 bytes
gzip -9.
Chromium and WebKit reject fetch with a bare TypeError and never attach a
cause, so the cause lookup could not trigger in Obsidian. Name what
errorDetail mirrors so its branches are not simplified away.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Deploying podnotes with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4bc910c
Status: ✅  Deploy successful!
Preview URL: https://24692d10.podnotes.pages.dev
Branch Preview URL: https://refactor-drop-openai-sdk.podnotes.pages.dev

View logs

@chhoumann

Copy link
Copy Markdown
Owner Author

Evidence

Failure notices, before and after

"Before" is the real openai 7.25.0 SDK driven with a stubbed fetch. "After" is what createTranscription's tests and the service tests assert. Every row is identical.

Whisper shows the same notice for every failure, before and after, because the API error never reached it:
Transcription failed: Transcription failed: all 1 audio chunk(s) failed or returned no text.

OpenAI diarization shows Transcription failed: OpenAI diarization failed for every chunk: <message>, where <message> is:

Failure <message> before (SDK) after (client)
401 {"error":{"message":M}} 401 Incorrect API key provided: sk-test. You can find your API key at https://platform.openai.com/account/api-keys. same
429 429 You exceeded your current quota, please check your plan and billing details. same
500 JSON error 500 The server had an error while processing your request. Sorry about that! same
502 text body 502 Bad Gateway same
503 empty body 503 status code (no body) same
Network failure Connection error. same
No response in 10 minutes Request timed out. same
fetch rejects "The request timed out." (WebKit) Request timed out. same
Edge JSON shapes (top-level message, empty or non-string message, error string, number, null) 500 Overloaded, 500 {"message":"","type":"server_error"}, 500 {"reason":"x"}, 500 "busy", 500 status code (no body), 500 null same

Cancellation: before and after, unloading PodNotes mid-upload aborts the fetch signal with AbortError: PodNotes was unloaded during transcription., writes no note and shows no notice.

Runtime check in Obsidian

Obsidian 1.13.7 on headless Linux (Xvfb), isolated vault and private HOME. The vault holds Audio/DEP1 Tone.mp3 (2 s, 16,552 bytes, SHA-256 17e2f720…). For each scenario the probe replaces window.fetch with a recorder, right-clicks the file in the file explorer, picks "Play with PodNotes", and runs "Transcribe current episode". The recorder encodes the request body with Chromium's Response, parses it back with formData(), and hashes the file part. The same scenarios ran on the master build (SDK) and on this branch's head bundle.

Scenario Uploads before → after Request, notice and note
Whisper, 200 1 → 1 identical
Diarization, 200 1 → 1 identical
Whisper, 401 3 → 3 identical
Diarization, 401 3 → 3 identical
Whisper, persistent 500 9 → 3 identical
Diarization, persistent 500 9 → 3 identical
Unload mid-upload 1 → 1 identical, signal aborted

Both builds sent POST https://api.openai.com/v1/audio/transcriptions, Authorization: Bearer <key>, and multipart fields model, file (DEP1 Tone.mp3, audio/mp3, 16,552 bytes, matching SHA-256), plus response_format=diarized_json and chunking_strategy=auto for diarization. The only header difference: the SDK also sent accept and seven x-stainless-* headers. A npm run dev bundle also loads, transcribes and reports no runtime errors.

Diarization 401 notice in Obsidian, this branch

Diarized transcript note written through the new client

Bundle size

npm run build, main.js, gzip measured with gzip -9 -n:

Commit Raw bytes Gzip bytes
master 8392951 677,800 177,286
transcribe through an in-house OpenAI client 312,953 (-53.8%) 98,757 (-44.3%)
drop the transcription credential cache 312,607 98,690
add Vite's dev/prod resolve condition 302,552 (-10,055) 95,457 (-3,233)
check only the fetch error message for timeouts (PR head) 302,480 95,420

The client commit removes 364,847 bytes, the size of openai 7.25.0 in the bundle. The resolve condition drops Svelte's dev-only error branches. The production bundle is byte-identical whether the condition is spelled production or development|production.

Commands

npm ci                  exit 0
npm run lint            exit 0
npm run format:check    exit 0, all 234 files formatted
npm run typecheck       exit 0
npm run build           exit 0, main.js 302.48 kB
npm run test            exit 0, svelte-check 0 errors 0 warnings, 80 files, 1185 tests passed
npm run docs:build      exit 0 (mkdocs 1.6.1 from docs/requirements.txt)

Run on the merge with master after npm ci. Each of the three change commits also typechecks and passes the suite on its own. package-lock.json differs from master only by the removed openai entries.

The openai SDK waited for the server-directed delay before retrying, so a
transient 429 with Retry-After: 4 recovered. createTranscription sends once
and the retry loops waited a fixed 1 s then 2 s, so the same 429 exhausted
all three attempts and the transcription failed.

A non-2xx response now carries the delay from retry-after-ms or
retry-after (seconds or HTTP date), bounded to 0-60 s as openai-node does.
Both the Whisper and OpenAI diarization loops wait that delay through
their existing abortable waits and fall back to the fixed backoff
otherwise.
…i-sdk

# Conflicts:
#	src/services/TranscriptionService.test.ts
Unit tests run in Node since #367, where the platform FormData encodes, so the request test can check the bytes that would go on the wire instead of the FormData object.
@chhoumann

Copy link
Copy Markdown
Owner Author

Fix round 1: server-directed retry delays

The independent verifier found that a transient 429 with Retry-After no longer recovered. The SDK honored that header inside its own retries, and the first version of this PR dropped it.

Change. A non-2xx response now throws an error that carries the server's retry delay. The delay is computed the way openai-node 7.25.0's retryRequest does it: retry-after-ms first, then Retry-After as seconds or an HTTP-date, and only values from 0 to 60 s count. Both retry loops wait that delay through their existing abortable waits, and otherwise 1 s, then 2 s. The client still sends once per attempt, and the attempt budget stays at 3.

Runtime, verifier scenario

The verifier's probe-extended.js, copied verbatim, ran in an isolated Obsidian 1.13.7 through the file explorer's "Play with PodNotes" and "Transcribe current episode". The server returns 429 with Retry-After: 4 until 4 s after the first request, then 200.

Build Whisper OpenAI diarization
master c2f7a52 sends at 0, 4,006 ms; note written sends at 0, 4,004 ms; note written
previous head ca91462 sends at 0, 1,001, 3,004 ms; failed; no note sends at 0, 1,004, 3,005 ms; failed; no note
this head sends at 0, 4,006 ms; note written sends at 0, 4,002 ms; note written

429 with Retry-After recovered on this head

The earlier seven scenarios were re-run on master and this head: Whisper and diarization success, 401, persistent 500, and unload mid-upload. All seven match, apart from 9 → 3 uploads on persistent 500 and the dropped telemetry headers.

Tests that fail on the previous head

  • TranscriptionService > Whisper waits out the server's Retry-After and recovers: expected sends at [0, 4000], received [0, 1000, 3000].
  • diarizeWithOpenAI > waits out the server's Retry-After and recovers the chunk: the old code rejects with OpenAI diarization failed for every chunk: 429 ....
  • unloading while Whisper waits out a Retry-After cancels the run and aborts while waiting out the server's Retry-After: the old code sends 3 times instead of once.
  • TranscriptionService > reads the current OpenAI key for each transcription: fails against a service that caches the first key (Bearer first-key instead of Bearer second-key).
  • A header table checks retry-after-ms, seconds, HTTP-date, both headers, over 60 s, negative, garbage, missing, and 0 against openai-node's rules. Eleven mutations of the delay rules and the waits each fail a named test.

Since #367 moved unit tests to Node, the client's request test now encodes the body with Response and parses it back. Appending the file as an unnamed Blob fails that test (blob instead of episode.mp3).

Commands on this head

npm ci                  exit 0
npm run lint            exit 0
npm run format:check    exit 0
npm run typecheck       exit 0
npm run build           exit 0, main.js 302,862 bytes (95,570 gzip -9 -n)
npm run test            exit 0, svelte-check 0 errors 0 warnings, 80 files, 1201 tests passed
npm run docs:build      exit 0

The branch is merged with master 39fa7ca (#364, #367). package-lock.json still differs from master only by the removed openai entries.

@chhoumann
chhoumann merged commit 07b2812 into master Oct 11, 2026
10 checks passed
@chhoumann
chhoumann deleted the refactor/drop-openai-sdk branch October 11, 2026 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant