Skip to content

fix(vad): exclude idle time from dynamic silence schedule - #3676

Merged
LauraGPT merged 1 commit into
mainfrom
codex/vad-idle-threshold-20260909
Sep 8, 2026
Merged

fix(vad): exclude idle time from dynamic silence schedule#3676
LauraGPT merged 1 commit into
mainfrom
codex/vad-idle-threshold-20260909

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Keep the dynamic FSMN silence schedule tied to the current detected speech segment instead of time spent waiting for speech. Related to #3302; this PR deliberately does not close that report.

  • Track total input samples and derive elapsed speech time from the VAD's absolute start timestamp. Initial and inter-utterance silence no longer ages the next utterance; fractional-millisecond packets retain their samples.
  • Preserve backdated starts and the last open segment when one feed returns several signals.
  • Reset stream state on the next feed after EOS, while preserving final state for the realtime service's tail fallback. Repeated finalize() is idempotent.
  • Leave the silence schedule, noise threshold, compensation, model weights and ASR decoding unchanged.

Verification

On ind-gpu8, using PyTorch 2.8.0+cu128 with CPU FSMN inference:

python -m pytest -q tests/test_dynamic_streaming_vad.py \
  tests/test_realtime_ws_service.py tests/test_realtime_ws_benchmark.py \
  tests/test_fsmn_vad_dynamic_silence.py tests/test_fsmn_vad_streaming_buffers.py

122 passed, no skips. The wrapper suite includes 16 tests. The original idle-duration defect and both EOS compatibility defects were reproduced as failing assertions before their respective fixes. git diff --check passes; independent static review found no remaining actionable issue.

Real-model controlled check: pinned FSMN snapshot v2.0.4, first 30 seconds of the reporter's YT-02.mp3, mono 16 kHz, 256 ms transport packets, three seconds of trailing silence. Compare the same clip with zero vs 60 seconds of preceding silence:

Wrapper No initial silence 60 seconds of initial silence
Before 2 segments 3 segments
After 2 segments 2 segments

After subtracting the prefixed silence, the fixed segment boundaries are [[0, 17690], [24140, 30280]] and [[-220, 17690], [24140, 30280]] ms. The 220 ms onset difference is VAD lookback into the prefixed silence; not every boundary is asserted identical.

Scope And Follow-Up

This uses the original playback file, not the reporter's captured microphone PCM, and does not measure ASR transcription accuracy. It establishes an idle-time-dependent VAD segmentation defect, not complete resolution of microphone truncation. The 42-second decode warning in the report remains a separate investigation. No arbitrary transport-packetization invariance or hardware fix is claimed; #3302 stays open for real microphone validation.

Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>
@LauraGPT
LauraGPT merged commit 0519852 into main Sep 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant