HostVoice v1.00.1

English | 中文

HostVoice is a local, NVIDIA CUDA-based real-time virtual microphone for software calls. It suppresses background noise and keeps only the enrolled main speaker while attenuating other people's voices.

Project: https://github.com/dboycht/hostvoice

Highlights

• NVIDIA NeMo TitaNet speaker enrollment and real-time target-speaker gating
• NVIDIA NeMo MarbleNet VAD plus energy VAD
• ONNX Runtime CUDA/TensorRT denoising with a built-in spectral-gate fallback
• Optional SpeechBrain SepFormer overlapping-speech separation on PyTorch CUDA
• Virtual microphone: VB-CABLE on Windows, PipeWire/PulseAudio on Linux
• Debug panel: device selection, microphone level tester, enrollment, gain/gate controls, logs

Pipeline

Microphone
  -> 48 kHz mono resampling
  -> High-pass / gain
  -> VAD (energy + MarbleNet)
  -> Denoise (ONNX CUDA/TensorRT or spectral gate)
  -> Optional source separation (SepFormer, CUDA)
  -> Target-speaker gate (TitaNet cosine similarity + smoothed gain)
  -> AGC / limiter
  -> Virtual microphone output

Quick Start

Windows

Double-click:

start.bat

Manual setup:

py -3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
pip install -r requirements-gpu.txt
powershell -ExecutionPolicy Bypass -File .\scripts\install_virtual_cable.ps1
python .\scripts\download_models.py --titanet --vad
python .\main.py

Linux / macOS

bash start.sh

Using an existing GPU environment

start_d2l_env.bat

Or:

powershell -NoProfile -ExecutionPolicy Bypass -File .\start_launcher.ps1 `
  -UsePython "D:\path\to\env\Scripts\python.exe" -NoDownload

Project Structure

main.py
start.py / start.bat / start_launcher.ps1 / start.sh
voicefocus/
ui/
scripts/
configs/default.yaml
requirements*.txt
docs/nvidia_tech_notes.md
tests/

Latency and Hardware

• Gate mode: about 40-80 ms; RTX 2060 or better recommended.
• Gate + ONNX CUDA denoise: about 50-120 ms.
• Experimental separation: 2-4 s buffering; RTX 3060 or better recommended.

Known Limitations

• Windows requires a signed virtual-audio driver (VB-CABLE).
• TitaNet identifies who is speaking; optional separation improves overlapping speech but is not perfect.
• Model weights require a one-time download; inference is local after caching.

---

中文

HostVoice v1.00.1

HostVoice 是一款基于 NVIDIA CUDA 的本地实时虚拟麦克风软件。它在软件通话中抑制环境噪声，并通过说话人识别只放行已注册主讲人的声音，衰减周围其他人声。

项目地址： https://github.com/dboycht/hostvoice

主要特性

• NVIDIA NeMo TitaNet 声纹注册与实时主讲人门控
• NVIDIA NeMo MarbleNet VAD + 能量 VAD
• ONNX Runtime CUDA/TensorRT 实时降噪，内置频谱门控回退
• 可选 SpeechBrain SepFormer 重叠人声分离（PyTorch CUDA）
• 虚拟麦克风：Windows 使用 VB-CABLE，Linux 使用 PipeWire/PulseAudio 自动创建
• 调试面板：设备选择、麦克风电平测试、声纹录制、门控/降噪参数、日志

处理流水线

物理麦克风
  -> 48 kHz 单声道重采样
  -> 高通 / 增益
  -> VAD（能量 + MarbleNet）
  -> 降噪（ONNX CUDA/TensorRT 或频谱门控）
  -> 可选人声分离（SepFormer, CUDA）
  -> 主讲人门控（TitaNet 余弦相似度 + 增益平滑）
  -> AGC / 限幅
  -> 虚拟麦克风输出

快速开始

Windows

双击：

start.bat

手动安装：

py -3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
pip install -r requirements-gpu.txt
powershell -ExecutionPolicy Bypass -File .\scripts\install_virtual_cable.ps1
python .\scripts\download_models.py --titanet --vad
python .\main.py

Linux / macOS

bash start.sh

使用已有 GPU 环境

start_d2l_env.bat

或：

powershell -NoProfile -ExecutionPolicy Bypass -File .\start_launcher.ps1 `
  -UsePython "D:\path\to\env\Scripts\python.exe" -NoDownload

目录结构

main.py
start.py / start.bat / start_launcher.ps1 / start.sh
voicefocus/
ui/
scripts/
configs/default.yaml
requirements*.txt
docs/nvidia_tech_notes.md
tests/

延迟与硬件建议

• 门控模式：约 40-80 ms；建议 RTX 2060 以上。
• 门控 + ONNX CUDA 降噪：约 50-120 ms。
• 实验性分离：需要 2-4 s 缓冲；建议 RTX 3060 以上。

已知限制

• Windows 需要签名虚拟声卡驱动（VB-CABLE）。
• TitaNet 解决“谁在说”；实验性分离可改善重叠说话场景，但无法完美分离任意混叠人声。
• 模型权重需首次联网下载；缓存后推理完全本地。