Skip to content

fix(voice): keep the in-flight final transcript when the participant switches mid-utterance - #7936

Open
BlueX888 wants to merge 1 commit into
livekit:mainfrom
BlueX888:fix/participant-switch-drops-transcript
Open

BlueX888 wants to merge 1 commit into
livekit:mainfrom
BlueX888:fix/participant-switch-drops-transcript

Conversation

@BlueX888

Copy link
Copy Markdown
Contributor

fix(voice): keep the in-flight final transcript when the participant switches mid-utterance

What breaks

RoomIO's transcription outputs finalize asynchronously: capture_text() queues a flush() task. When the local participant switches mid-utterance, the switch replaces the represented participant/track state before the queued flush runs, so the user's in-flight final transcript is published under the new participant:

  • stream sink: the final carries the new participant's identity/track and a fresh segment id (verified: sender=user-b, TR_b, chunks=[''] — the original text lost),
  • legacy sink: identity is new while the segment/track attributes are the old ones (a half-attributed segment).

Fix

Both sinks now flush before replacing the participant identity/track, and each flush snapshots the attribution it must publish with:

  • stream sink: flush() snapshots (sender_identity, track_id, segment_id, latest_text) into _flush_task; _create_text_writer accepts explicit sender/segment overrides, so the deferred publish never reads reset/new-participant state.
  • legacy sink: flush() snapshots (represented_by/identity, track_id) into _publish_transcription; set_participant clears _track_id and always re-resolves the new participant's mic track, so the old track_sid cannot leak.

Tests

tests/test_room_io.py::test_participant_switch_mid_utterance_keeps_the_final_transcript (deterministic _FakeRoom/_FakeWriter harness): captures an STT interim for user-a, switches to user-b before the final, then aclose() awaits both flush tasks.

  • On base: fails — the stream final is attributed to user-b (assert 'user-b' == 'user-a').
  • With the fix: exactly one stream final and one legacy final, each with A's text, identity, original segment id and TR_a; no final under the new participant. test_room_io.py 61 passed, transcription group 483, test_agent_session 110; ruff/mypy clean on room_io.

Independent review probes covered A→B, A→B→C, A→None and A→B→A: exactly one final per captured segment with correct identities/ids (no double flushing); the legacy no-mic path and publish_on_behalf resolution are preserved. Prior art: closed #7531 fixed only the legacy sink, so the primary stream-sink bug remained. No overlap with open PRs #7933 / #7928 (different files).

… switches

_ParticipantStreamTranscriptionOutput.set_participant replaced the participant
identity and track, then scheduled flush() and synchronously reset the segment
state. The flush task later read the new identity, track and segment id, so an
in-flight final was published as an empty packet under the new participant with
a fresh segment id, and the previous participant's segment was never finalized.
The non-delta user transcription sink takes that path on a warm-transfer switch
mid-utterance.

The legacy sink had the same deferred-state race, and kept the previous
participant's track because it only re-resolved the track when _track_id was
empty; its final went out as the new identity on the old track.

Finalize the open segment before replacing the participant, and snapshot the
segment id, identity, track and latest text into the flush task so the deferred
publish uses the state the segment was captured under. Always re-resolve the
legacy track for the newly linked participant.

Add a regression test that switches participants while an STT interim is open
and asserts both sinks close the previous segment exactly once with the
original sender, track and segment id.
@BlueX888
BlueX888 requested a review from a team as a code owner October 18, 2026 23:23

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

Comment on lines +598 to +600
# finalize the open segment for the participant it was captured for before the
# identity and track are replaced; flush() snapshots them for its publish task
self.flush()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Participant switches mix two users' transcripts

When set_participant runs during the first capture_text stream request, flush() cannot close its still-unassigned writer. If that capture resumes before the next chunk, the new user's text enters the old user's transcript.

Learn more

A delta transcript keeps one open text writer for all its chunks. capture_text marks the segment as capturing before awaiting writer creation, then assigns _writer after the request completes. If a participant switches during that await, flush() sees _writer=None, schedules a close that does nothing, and clears the capturing flag. The pending request can then assign the old writer; the next capture resets segment state but reuses that writer, mixing participants.

Example: A's first chunk awaits stream_text. The room switches to B, and the flush runs without a writer. A's request finishes; B's first chunk then writes to A's open stream instead of starting B's stream.

Recommended fix: Serialize participant switches with outstanding capture_text operations, or retain and close a pending writer after its creation before allowing the new participant's capture to use _writer. Cover a switch while stream_text is suspended in a delta-stream test.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant