Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@
| Distinguish speaker vectors, anonymous labels and emotion tags | [Speakers and emotion tags](speaker_emotion.md) / [中文](speaker_emotion_zh.md) |
| Transcribe and diarize with third-party MOSS | [MOSS](moss_transcribe_diarize.md) / [中文](moss_transcribe_diarize_zh.md) |
| Accelerate with the FunASR vLLM split engine | [vLLM](vllm_guide.md) / [中文](vllm_guide_zh.md) |
| Evaluate native vLLM serving | [Validation record](vllm_native_funasr_validation.md) |
| Evaluate official native vLLM serving | [Official checkpoint validation](vllm_official_native_validation.md) / [中文](vllm_official_native_validation_zh.md) |
| Inspect the historical community native vLLM run (2026-08-13) | [Historical community record](vllm_native_funasr_validation.md) |
| Deploy llama.cpp, TensorRT, Docker or Kubernetes | [Deployment manuals](https://www.funasr.com/en/deploy/) |
| Choose a service entry point and runtime SDK | [Runtime guide](../runtime/readme.md) / [中文](../runtime/readme_cn.md) |
| Select a development or serving container | [Docker](installation/docker.md) / [中文](installation/docker_zh.md) |
Expand Down
28 changes: 26 additions & 2 deletions docs/model_selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Switch to Paraformer when your workload is Mandarin-only and you want character-
| Fast multilingual private transcription | SenseVoice-Small | Strong default with ASR, emotion tags, audio event tags, and CPU viability. | [README quick start](../README.md#quick-start) |
| Mandarin production ASR | Paraformer-Large | Mature Chinese ASR path with VAD and punctuation. | [Tutorial](./tutorial/README.md) |
| English-only route in the OpenAI API example | `paraformer-en` alias | Smaller English route for API compatibility checks. | [OpenAI API example](../examples/openai_api/) |
| LLM-based ASR or Chinese/English/Japanese + dialect experiments | Fun-ASR-Nano | LLM-based model path; use vLLM when decoder throughput matters. | [vLLM guide](./vllm_guide.md) |
| LLM-based ASR or Chinese/English/Japanese + dialect experiments | Fun-ASR-Nano | Evaluate the Python path first; split-engine and native vLLM use different loading contracts. | [Choose a vLLM path](#vllm-checkpoint-paths) |
| Offline long-form ASR with anonymous diarization | MOSS-Transcribe-Diarize | One offline request returns transcription, timestamps, and per-recording anonymous speaker labels; it does not identify known people and needs no external VAD or speaker model. | [MOSS deployment guide](./moss_transcribe_diarize.md) |
| Live captions or call-center streams | Runtime WebSocket service | Designed for long-lived streaming sessions and partial results. | [Runtime service docs](../runtime/readme.md) |
| Batch archive processing | SenseVoice-Small or Paraformer-Large | Stable offline transcription path; caller owns manifests, retries, and logs. | [Batch ASR example](../examples/batch_asr_improved.py) |
Expand All @@ -56,6 +56,13 @@ The `examples/openai_api` server exposes short aliases so application teams do n
- **`paraformer`** uses `paraformer-zh` with VAD and punctuation for a Mandarin-oriented route.
- **`paraformer-en`** uses `paraformer-en` with VAD for English transcription in OpenAI-style clients.
- **`fun-asr-nano`** uses `FunAudioLLM/Fun-ASR-Nano-2512` for evaluating Chinese, English, Japanese, and Chinese dialect/accent coverage. Select a compatible runtime when evaluating vLLM acceleration.
- **`moss-transcribe-diarize`** uses the third-party `OpenMOSS-Team/MOSS-Transcribe-Diarize` model for offline transcription and anonymous per-recording speaker labels. Prepare its separate dependencies and reviewed remote code using the [MOSS guide](./moss_transcribe_diarize.md); request `verbose_json` for structured segments. It does not require an external VAD/speaker model and does not identify known people.

These aliases describe [the example server](../examples/openai_api/server.py),
which loads `AutoModel`. They do not configure native vLLM or automatically select
`AutoModelVLLM`. The packaged `funasr-server` has a separate loader and backend
selection; do not copy an alias or a performance result between services without
checking the corresponding [HTTP guide](../examples/openai_api/README.md).

The example HTTP service cleans both top-level `text` and segment `text` in
`verbose_json`; that format does not restore emotion/event tags. If you need
Expand All @@ -82,10 +89,27 @@ For SDK, JavaScript, workflow, Postman, OpenAPI, Docker, and Kubernetes paths, s
| Repeatable local container demo | Docker Compose API | CPU-first smoke test; adapt the image before using CUDA. |
| Internal cluster service | Kubernetes API template | Private `ClusterIP`, persistent model cache, `/health` probes, and port-forward smoke test. |
| Live audio | Runtime WebSocket service | Validate chunk size, VAD, endpointing, reconnects, and client backpressure with real audio. |
| LLM-based ASR throughput | vLLM path for Fun-ASR-Nano | vLLM accelerates autoregressive decoding; it does not apply to non-autoregressive Paraformer. |
| LLM-based ASR throughput | Choose split-engine or native vLLM below | Match the checkpoint, loading API and tested environment; this is not a Paraformer backend. |

See the [deployment matrix](./deployment_matrix.md) when you are choosing between these paths.

<a id="vllm-checkpoint-paths"></a>

## Choose the vLLM Checkpoint and Interface

| Path | Checkpoint and interface | Read next |
| --- | --- | --- |
| FunASR split-engine | Base `FunAudioLLM/Fun-ASR-Nano-2512` assets through `AutoModelVLLM`; FunASR handles the audio side and vLLM the decoder. | [Split-engine preparation and limits](./vllm_guide.md) |
| Official native vLLM | Converted `FunAudioLLM/Fun-ASR-Nano-2512-vllm` snapshot through vLLM's native model implementation and `/v1/audio/transcriptions`. Not an `AutoModelVLLM` load. | [Official functional validation](./vllm_official_native_validation.md) |
| Historical community native vLLM | Community `allendou/Fun-ASR-Nano-2512-vllm`, tested on 2026-08-13. Its timings belong to that checkpoint and environment. | [Historical community record](./vllm_native_funasr_validation.md) |

The official record pins a model revision and an existing environment; it is not
a clean-install recipe, a sustained-load benchmark, or proof of `/v1/realtime`
streaming. Do not reuse the historical community timings for the official model.
For MOSS, follow its own deployment guide: the Nano checkpoints and validation
above do not establish MOSS runtime compatibility. Choose model, checkpoint,
interface and environment together before testing your own workload.

## Benchmark before committing

Do not choose a model from a single clean demo file. Use a small representative set first:
Expand Down
23 changes: 22 additions & 1 deletion docs/model_selection_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ result = model.generate(input="meeting.wav")
| 高速な多言語プライベート文字起こし | SenseVoice-Small | ASR、感情タグ、音声イベントタグ、CPU/GPU の扱いやすさがそろった標準ルート。 | [README quick start](../README_ja.md#クイックスタート) |
| 中国語中心の本番 ASR | Paraformer-Large | VAD と句読点復元を組み合わせた成熟した中国語 ASR ルート。 | [Tutorial](./tutorial/README.md) |
| OpenAI API 例の英語ルート | `paraformer-en` alias | OpenAI-style client で互換性を確認しやすい軽量な英語ルート。 | [OpenAI API example](../examples/openai_api/README_ja.md) |
| LLM-based ASR や中英日 + 中国語方言・地域アクセントの評価 | Fun-ASR-Nano | LLM-based モデル。decoder throughput が重要なら vLLM を使います。 | [vLLM guide](./vllm_guide.md) |
| LLM-based ASR や中英日 + 中国語方言・地域アクセントの評価 | Fun-ASR-Nano | Python で評価してから、checkpoint と interface に合わせて vLLM の経路を選びます。 | [vLLM の経路](#vllm-checkpoint-paths) |
| オフライン長時間 ASR と匿名話者ラベル | MOSS-Transcribe-Diarize | 1 回のオフライン request で文字起こし、timestamps、録音内の匿名話者ラベルを返します。既知人物の識別ではなく、外部 VAD / speaker model も不要です。 | [MOSS deployment guide](./moss_transcribe_diarize.md) |
| ライブ字幕やコールセンターストリーム | Runtime WebSocket service | 長時間接続、部分結果、エンドポイント検出に向いたランタイム。 | [Runtime service docs](../runtime/readme.md) |
| Whisper / cloud ASR からの移行 | SenseVoice-Small で baseline を作り、必要に応じて比較 | まず強い標準ルートで評価してから、用途別に詰めるのが安全です。 | [Migration guide](./migration_from_whisper.md) |
Expand All @@ -42,6 +42,12 @@ result = model.generate(input="meeting.wav")
- **`paraformer`**: `paraformer-zh` に VAD と句読点復元を組み合わせた中国語向けの経路です。
- **`paraformer-en`**: `paraformer-en` と VAD を使う、OpenAI-style client 向けの英語文字起こしです。
- **`fun-asr-nano`**: `FunAudioLLM/Fun-ASR-Nano-2512` による中英日・中国語方言/地域アクセントの評価経路です。vLLM acceleration を試す場合は互換性のある runtime を選んでください。
- **`moss-transcribe-diarize`**: 第三者の `OpenMOSS-Team/MOSS-Transcribe-Diarize` によるオフライン文字起こしと録音内の匿名話者ラベルです。[MOSS guide(英語)](./moss_transcribe_diarize.md)で専用の依存関係と remote code を確認し、構造化 segment には `verbose_json` を指定します。外部 VAD / speaker model は不要で、既知人物の識別ではありません。

ここでの alias は `AutoModel` を読み込む[サンプル server](../examples/openai_api/server.py)のものです。
native vLLM や `AutoModelVLLM` を自動選択する設定ではありません。
パッケージの `funasr-server` は別の loader / backend 選択を持つため、
サービス間で alias や性能結果をそのまま流用しないでください。

この HTTP サンプルはトップレベルの `text` と `verbose_json` の各 segment の
`text` を整形するため、形式を変えても感情/イベントタグは復元されません。
Expand All @@ -57,6 +63,21 @@ python examples/openai_api/smoke_test.py --base-url http://localhost:8000 --mode

SDK、JavaScript、workflow、Postman、OpenAPI、Docker、Kubernetes は [OpenAI API example](../examples/openai_api/README_ja.md) から始めてください。

<a id="vllm-checkpoint-paths"></a>

## vLLM の checkpoint と interface

| 経路 | checkpoint と interface | 次の資料 |
| --- | --- | --- |
| FunASR split-engine | 基本の `FunAudioLLM/Fun-ASR-Nano-2512` を `AutoModelVLLM` で読み込み、音声側は FunASR、decoder は vLLM が処理します。 | [Split-engine(英語)](./vllm_guide.md) |
| 公式 native vLLM | 変換済み `FunAudioLLM/Fun-ASR-Nano-2512-vllm` を vLLM の native 実装で読み込み、`/v1/audio/transcriptions` を利用します。`AutoModelVLLM` ではありません。 | [公式機能検証(英語)](./vllm_official_native_validation.md) |
| 過去の community native vLLM | `allendou/Fun-ASR-Nano-2512-vllm`、2026-08-13 の検証です。時間計測は当時の checkpoint と環境に限定されます。 | [過去の community 記録](./vllm_native_funasr_validation.md) |

公式記録は固定 revision と既存環境での機能検証であり、新規インストール手順、
持続負荷の benchmark、`/v1/realtime` の streaming 検証ではありません。
過去の community 計測を公式モデルに付け替えないでください。
MOSS は専用ガイドに従ってください。Nano の checkpoint と検証は MOSS の互換性を証明しません。

## ベンチマークしてから決める

きれいな demo 音声 1 つだけでモデルを決めないでください。まず小さな代表セットで確認します。
Expand Down
23 changes: 22 additions & 1 deletion docs/model_selection_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ result = model.generate(input="meeting.wav")
| 빠른 다국어 프라이빗 전사 | SenseVoice-Small | ASR, 감정 태그, 음성 이벤트 태그, CPU/GPU 사용성이 균형 잡힌 기본 경로입니다. | [README quick start](../README_ko.md#빠른-시작) |
| 중국어 중심 프로덕션 ASR | Paraformer-Large | VAD와 문장부호 복원을 함께 쓰는 성숙한 중국어 ASR 경로입니다. | [Tutorial](./tutorial/README.md) |
| OpenAI API 예제의 영어 경로 | `paraformer-en` alias | OpenAI-style client에서 호환성을 확인하기 쉬운 가벼운 영어 경로입니다. | [OpenAI API example](../examples/openai_api/README_ko.md) |
| LLM-based ASR 또는 중영일 + 중국어 방언/지역 억양 평가 | Fun-ASR-Nano | LLM-based 모델입니다. decoder throughput이 중요하면 vLLM을 사용합니다. | [vLLM guide](./vllm_guide.md) |
| LLM-based ASR 또는 중영일 + 중국어 방언/지역 억양 평가 | Fun-ASR-Nano | Python에서 평가한 뒤 checkpoint와 interface에 맞는 vLLM 경로를 선택합니다. | [vLLM 경로 선택](#vllm-checkpoint-paths) |
| 오프라인 장시간 ASR 및 익명 화자 라벨 | MOSS-Transcribe-Diarize | 한 번의 오프라인 request로 전사, timestamps, 녹음 내 익명 화자 라벨을 반환합니다. 알려진 인물을 식별하지 않으며 외부 VAD / speaker model도 필요하지 않습니다. | [MOSS deployment guide](./moss_transcribe_diarize.md) |
| 라이브 자막 또는 콜센터 스트림 | Runtime WebSocket service | 장시간 연결, 부분 결과, endpointing에 맞춘 런타임입니다. | [Runtime service docs](../runtime/readme.md) |
| Whisper / cloud ASR에서 전환 | SenseVoice-Small로 baseline을 만들고 필요하면 비교 | 강한 기본 경로로 먼저 평가한 뒤 용도별로 조정하는 편이 안전합니다. | [Migration guide](./migration_from_whisper.md) |
Expand All @@ -42,6 +42,12 @@ result = model.generate(input="meeting.wav")
- **`paraformer`**: `paraformer-zh`에 VAD와 문장부호 복원을 결합한 중국어 경로입니다.
- **`paraformer-en`**: `paraformer-en`과 VAD를 사용하는 OpenAI-style client용 영어 전사입니다.
- **`fun-asr-nano`**: `FunAudioLLM/Fun-ASR-Nano-2512`로 중영일·중국어 방언/지역 억양을 평가합니다. vLLM acceleration을 시험할 때는 호환되는 runtime을 선택하세요.
- **`moss-transcribe-diarize`**: 서드파티 `OpenMOSS-Team/MOSS-Transcribe-Diarize`의 오프라인 전사와 녹음 내 익명 화자 라벨입니다. [MOSS guide(영문)](./moss_transcribe_diarize.md)에서 전용 의존성과 remote code를 검토하고, 구조화된 segment에는 `verbose_json`을 요청하세요. 외부 VAD / speaker model이 필요 없으며 알려진 인물을 식별하지 않습니다.

여기서 설명하는 alias는 `AutoModel`을 로드하는 [예제 server](../examples/openai_api/server.py)의 설정입니다.
native vLLM이나 `AutoModelVLLM`을 자동으로 선택하지 않습니다.
패키지의 `funasr-server`는 별도 loader / backend 선택 로직을 사용하므로,
서비스 사이에서 alias나 성능 결과를 그대로 재사용하지 마세요.

이 HTTP 예제는 최상위 `text`와 `verbose_json`의 각 segment `text`를 정리하므로,
형식을 바꿔도 감정/이벤트 태그가 복원되지 않습니다. 원래 태그가 필요하면 Python SDK를
Expand All @@ -57,6 +63,21 @@ python examples/openai_api/smoke_test.py --base-url http://localhost:8000 --mode

SDK, JavaScript, workflow, Postman, OpenAPI, Docker, Kubernetes는 [OpenAI API example](../examples/openai_api/README_ko.md)에서 시작하세요.

<a id="vllm-checkpoint-paths"></a>

## vLLM checkpoint와 interface 선택

| 경로 | checkpoint와 interface | 다음 문서 |
| --- | --- | --- |
| FunASR split-engine | 기본 `FunAudioLLM/Fun-ASR-Nano-2512`를 `AutoModelVLLM`으로 로드합니다. FunASR은 오디오 부분, vLLM은 decoder를 처리합니다. | [Split-engine(영문)](./vllm_guide.md) |
| 공식 native vLLM | 변환된 `FunAudioLLM/Fun-ASR-Nano-2512-vllm`을 vLLM의 native 구현으로 로드하고 `/v1/audio/transcriptions`를 사용합니다. `AutoModelVLLM` 로드가 아닙니다. | [공식 기능 검증(영문)](./vllm_official_native_validation.md) |
| 과거 community native vLLM | `allendou/Fun-ASR-Nano-2512-vllm`, 2026-08-13 검증입니다. 측정 시간은 당시 checkpoint와 환경에만 해당합니다. | [과거 community 기록](./vllm_native_funasr_validation.md) |

공식 기록은 고정 revision과 기존 환경에서의 기능 검증이며, 신규 설치 절차,
지속 부하 benchmark 또는 `/v1/realtime` streaming 검증이 아닙니다.
과거 community 측정 시간을 공식 모델의 결과로 사용하지 마세요.
MOSS는 별도 가이드를 따르세요. Nano checkpoint와 검증은 MOSS 호환성을 입증하지 않습니다.

## 벤치마크 후 결정하기

깨끗한 demo 오디오 하나만 보고 모델을 정하지 마세요. 먼저 작은 대표 세트로 확인합니다.
Expand Down
Loading
Loading