fix(vad): exclude idle time from dynamic silence schedule - #3676
Merged
Conversation
Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep the dynamic FSMN silence schedule tied to the current detected speech segment instead of time spent waiting for speech. Related to #3302; this PR deliberately does not close that report.
finalize()is idempotent.Verification
On ind-gpu8, using PyTorch 2.8.0+cu128 with CPU FSMN inference:
122 passed, no skips. The wrapper suite includes 16 tests. The original idle-duration defect and both EOS compatibility defects were reproduced as failing assertions before their respective fixes.
git diff --checkpasses; independent static review found no remaining actionable issue.Real-model controlled check: pinned FSMN snapshot
v2.0.4, first 30 seconds of the reporter'sYT-02.mp3, mono 16 kHz, 256 ms transport packets, three seconds of trailing silence. Compare the same clip with zero vs 60 seconds of preceding silence:After subtracting the prefixed silence, the fixed segment boundaries are
[[0, 17690], [24140, 30280]]and[[-220, 17690], [24140, 30280]]ms. The 220 ms onset difference is VAD lookback into the prefixed silence; not every boundary is asserted identical.Scope And Follow-Up
This uses the original playback file, not the reporter's captured microphone PCM, and does not measure ASR transcription accuracy. It establishes an idle-time-dependent VAD segmentation defect, not complete resolution of microphone truncation. The 42-second decode warning in the report remains a separate investigation. No arbitrary transport-packetization invariance or hardware fix is claimed; #3302 stays open for real microphone validation.