Skip to content

fix(asr): 完善 ElevenLabs Scribe 接入并解决 beta 冲突 - #800

Closed
H-Chris233 wants to merge 6 commits into
Open-Less:betafrom
H-Chris233:agent/pr-797-elevenlabs-fix
Closed

H-Chris233 wants to merge 6 commits into
Open-Less:betafrom
H-Chris233:agent/pr-797-elevenlabs-fix

Conversation

@H-Chris233

@H-Chris233 H-Chris233 commented Jul 14, 2026 •

Copy link
Copy Markdown
Collaborator

User description

变更

  • 将 PR feat(asr): 新增 ElevenLabs Scribe 语音转录供应商 #797 的 ElevenLabs Scribe 接入重放到最新 beta,解决全部合并冲突。
  • 完成听写、QA、Overlay QA、重新转录、配置验证和模型列表接入。
  • 增加共享端点安全策略、DNS 解析固定、连接/动态超时和响应大小限制。
  • 修复转写取消竞态,以及 provider 切换时自定义 endpoint/model 的跨 provider 写入问题。
  • 更新隐私提示和 provider 分类回归测试。

根因

原 PR 基于旧版 beta,与当前 beta 在 ASR、coordinator 和设置页面存在冲突;同时自定义端点实际请求路径未完全复用安全校验,QA 转写和设置凭据写入存在竞态。

验证

  • cargo check --locked --lib
  • ElevenLabs 定向测试:11/11
  • provider 分类测试:3/3
  • npm.cmd run build
  • git diff --check

该 PR 替代 #797。原 PR 的 head 属于无当前账号写权限的 Billmvp73/openless,因此无法直接更新其源分支。


PR Type

Enhancement, Documentation


Description

  • Add ElevenLabs Scribe batch ASR provider

  • Introduce provider-scoped credential storage

  • Extract shared HTTP endpoint validation module

  • Update frontend presets, i18n, and privacy notice


Diagram Walkthrough

flowchart LR
  A["New ElevenLabs client"] --> B["16kHz PCM → WAV"]
  B --> C["POST /v1/speech-to-text multipart"]
  C --> D["xi-api-key header, tag_audio_events=false"]
  D --> E["Extract text (single/multi‑channel)"]
  A --> F["Integrated into dictation, QA, retranscribe, cancel"]
  F --> G["ActiveAsr::ElevenLabs kind dispatch"]
  G --> H["Validation, model list, credential checks"]
  H --> I["Frontend preset, i18n, privacy notice"]
  I --> J["Endpoint security: validate + DNS pinning"]
Loading

File Walkthrough

Relevant files
Enhancement
15 files
elevenlabs.rs
Add ElevenLabs Scribe ASR client                                                 
+531/-0 
mod.rs
Register elevenlabs submodule                                                       
+2/-0     
credentials.rs
Accept optional provider in credential commands                   
+32/-4   
providers.rs
Add ElevenLabs validation and model list                                 
+53/-11 
coordinator.rs
Handle ElevenLabs ASR in coordinator                                         
+43/-84 
asr_wiring.rs
Add is_elevenlabs_provider and build wiring                           
+11/-0   
dictation.rs
Wire ElevenLabs into dictation sessions                                   
+36/-0   
qa_session.rs
Wire ElevenLabs into QA sessions                                                 
+61/-0   
resources.rs
Cancel ElevenLabs ASR                                                                       
+1/-0     
endpoint_security.rs
Extract shared HTTP endpoint validation                                   
+109/-0 
lib.rs
Declare endpoint_security module                                                 
+1/-0     
credentials.rs
Add get/set_for_asr_provider methods                                         
+22/-0   
asr-credentials.ts
Forward provider param in credential functions                     
+4/-4     
ProvidersSection.tsx
Add provider‑scoped credential fields and notice                 
+31/-17 
shared.tsx
Add ElevenLabs preset entry                                                           
+1/-0     
Documentation
5 files
en.ts
Add ElevenLabs preset and update privacy strings                 
+5/-3     
ja.ts
Translate ElevenLabs preset and privacy strings                   
+5/-3     
ko.ts
Translate ElevenLabs preset and privacy strings                   
+5/-3     
zh-CN.ts
Translate ElevenLabs preset and privacy strings                   
+5/-3     
zh-TW.ts
Translate ElevenLabs preset and privacy strings                   
+5/-3     

Billmvp73 and others added 6 commits July 13, 2026 00:32
Add ElevenLabs Speech-to-Text (Scribe) as a batch ASR provider.

ElevenLabs uses `POST /v1/speech-to-text` with multipart/form-data
(`model_id` + `file`) and the `xi-api-key` header — not OpenAI's Bearer
`/audio/transcriptions` protocol — so it gets a dedicated client
(`asr/elevenlabs.rs`) modeled on the MiMo batch client rather than riding
on WhisperBatchASR. `tag_audio_events=false` keeps bracketed non-speech
events out of dictation transcripts.

Wiring:
- ActiveAsr::ElevenLabs + ActiveAsrProviderKind::ElevenLabs, dictation /
  QA / retranscribe transcribe arms, cancel arm, credential reader.
- provider validation + model list (scribe_v1 / scribe_v2).
- credentials "configured" check (api key + endpoint + model).
- frontend preset, Overview label, en/zh-CN/zh-TW/ja/ko i18n.

Verified the wire protocol end-to-end against the live ElevenLabs API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scribe_v1 is deprecated and is removed from the ElevenLabs API on
2026-07-09, so shipping it as the default would break shortly. scribe_v2
is the current recommended model (higher accuracy, 99 languages, ~40%
cheaper, multi-language in one file). Update the Rust DEFAULT_MODEL, the
frontend preset, and the validation/model-list fallback accordingly;
keep scribe_v1 selectable in the model list for now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
active_asr_provider_kind's test locked bailian/whisper/mimo/volcengine
but not ElevenLabs, so a regression in is_elevenlabs_provider or its
ordering in the dispatch chain would go uncaught. Add the assertion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

797 - Partially compliant

Compliant requirements:

  • 新增 ElevenLabs Scribe v2 批式语音转录供应商
  • 默认 endpoint 为 https://api.elevenlabs.io/v1,默认模型为 scribe_v2;支持自定义 endpoint/model
  • 自定义 endpoint 必须使用 HTTPS;仅为本地开发保留 loopback HTTP
  • 请求禁止自动跟随重定向
  • 服务端错误只展示 HTTP 状态和本地白名单错误码
  • 覆盖普通听写、悬浮问答、历史重转录、取消、凭据检查、连接验证和 provider dispatch
  • 设置页新增 ElevenLabs 预设,并补齐 en / zh-CN / zh-TW / ja / ko 的供应商名称
  • 新增请求格式、单/多声道响应解析、空缓冲、provider-kind dispatch 等回归测试
  • 纯增量;不修改其他 ASR provider 的默认值、凭据格式或行为

Non-compliant requirements:

  • 不包含:Scribe v2 Realtime WebSocket、keyterm prompting、entity detection、speaker diarization 等高级能力(明确不在本次范围内)
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

@H-Chris233

Copy link
Copy Markdown
Collaborator Author

已通过 maintainer edit 将同一提交推回原始 PR #797;本 PR 为重复替代 PR,关闭。

@H-Chris233 H-Chris233 closed this Jul 14, 2026
@H-Chris233
H-Chris233 deleted the agent/pr-797-elevenlabs-fix branch August 3, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants