fix(agent): stop cutting the transcript at the 800th word - #256
Open
EtienneLescot wants to merge 2 commits into
Open
fix(agent): stop cutting the transcript at the 800th word#256EtienneLescot wants to merge 2 commits into
EtienneLescot wants to merge 2 commits into
Conversation
`getTranscript` sliced at 800 segments under a comment reading "segments only — words would blow the context", written believing a segment was a phrase. On the production path a segment IS one word: whisper's word timings are mapped one-to-one in src/lib/captioning/transcribe.ts, and the real fixture has 129 words for 129 segments. So the cap cut at the 800th WORD. At a normal speaking rate that is the fifth minute, and nothing in the payload said so — the model read a sixth of a half-hour recording, trimmed the silences it could see, and reported the job done. Asking it afterwards does not help either: the history sent back carries message text, not tool results, so the next turn re-reads the same 800 words. A whole 30-minute transcript is ~285k characters, ~70k tokens — well inside every model this app talks to. The cap was a guess, not a measurement, so it is gone rather than raised. If a recording ever does approach a window, the fix is to know the window; there is no per-model context budget in the app today, and picking another number here would only move the silence. The workbench already carried a `longTranscript` fixture documenting this defect and never wired it to a scenario. Its comment is now true instead of a to-do.
📝 WalkthroughWalkthrough
ChangesTranscript retrieval
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
3 tasks
`ai-agent.md` still announced "up to 800 transcript segments" — the number this branch removed, and the one that quietly cut a half-hour recording at its fifth minute. Reference that repeats a limit the code dropped is how the limit gets re-added.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@electron/ai-edition/agent-tools.test.ts`:
- Around line 215-221: Update the generated segment objects in the transcript
fixture around the segments array to include the required wordIds field, using
an empty array for every segment while preserving the existing id, timing, kind,
and text values.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 257d65f3-b1c4-4668-8318-3fb46ddd31da
📒 Files selected for processing (4)
electron/ai-edition/agent-tools.test.tselectron/ai-edition/agent-tools.tstechnical-documentation/architecture/ai-agent.mdworkbench/lib/fixtures.ts
Comment on lines
+215
to
+221
| const segments = Array.from({ length: 4000 }, (_, i) => ({ | ||
| id: `seg_${i}`, | ||
| kind: "speech" as const, | ||
| startSec: i * 0.45, | ||
| endSec: i * 0.45 + 0.4, | ||
| text: `mot${i}`, | ||
| })); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 3 '\bwordIds\b' electron/ai-edition workbench/lib --glob '*.ts'Repository: getopenscreen/openscreen
Length of output: 4739
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the relevant test file and any AxcutDocument/type declarations used by the failing line.
sed -n '1,260p' electron/ai-edition/agent-tools.test.ts
printf '\n--- type references in files ---\n'
rg -n 'interface AxcutDocument|type AxcutDocument|AxcutDocument|documentSchema|wordIds' --glob '*.ts' .Repository: getopenscreen/openscreen
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- schema transcript definitions --- '
rg -n -C 4 'AxcutTranscriptSegment|wordIds|transcript' src/lib/ai-edition/schema.ts
printf '\n--- schema file outline around definitions ---\n'
nl -ba src/lib/ai-edition/schema.ts | sed -n '1,260p'
printf '\n--- read-only structural check: generated segments lack wordIds ---\n'
python3 - <<'PY'
from pathlib import Path
text = Path("electron/ai-edition/agent-tools.test.ts").read_text()
needle = 'const segments = Array.from({ length: 4000 }, (_, i) => ({'
idx = text.index(needle)
block = text[idx:text.find("\n\t\t});", idx)]
print("wordIds" in block)
print(block)
PYRepository: getopenscreen/openscreen
Length of output: 330
🏁 Script executed:
#!/bin/bash
set -euo pipefail
fd -a 'schema\.ts$' . | sed 's#^\./##'
printf '%s\n' '--- schema path candidates ---'
for f in $(fd 'schema\.ts$' .); do
echo "FOUND $f"
done
printf '%s\n' '--- references to schema module ---'
rg -n 'from ["'\''][^"'\'']*ai-edition/schema' --glob '*.ts' .Repository: getopenscreen/openscreen
Length of output: 4841
Add wordIds to each generated transcript segment.
This is an AxcutDocument, so transcript segments need the schema field shown in the other transcript fixtures. Add wordIds: [] to the generated objects.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/ai-edition/agent-tools.test.ts` around lines 215 - 221, Update the
generated segment objects in the transcript fixture around the segments array to
include the required wordIds field, using an empty array for every segment while
preserving the existing id, timing, kind, and text values.
Source: Linters/SAST tools
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ask the agent to remove the silences from a 30-minute recording and it works on the first five minutes, then reports the job done.
getTranscriptsliced at 800 segments:The comment is inverted. On the production path a segment is one word —
src/lib/captioning/transcribe.tsmaps whisper's word timings one-to-one, and the real fixture has 129 words for 129 segments. So the cap cut at the 800th word, not the 800th phrase.The arithmetic lands on the reported symptom: 129 words / 66.15 s = 1.95 words/s puts word 800 at 6 min 50 s; at 160 wpm it is 300 s exactly. Five minutes out of thirty.
And it was silent. The payload is
{assetId, language, segments}— notruncated, nototalSegments— while the tool description promises the transcript for the asset. The model had no way to know it was reading a fifth of the recording, so it trimmed what it saw and called it finished.getCursorTracknext door does report truncation, with a comment saying that is how the model learns it.Saying "continue" does not recover it: the history sent back to the model carries message text, not tool results, so the next turn re-reads the same 800 words.
Removed rather than raised. A whole 30-minute transcript is ~285k characters, ~70k tokens — well inside every model this app talks to. The cap was a guess, not a measurement. If a recording ever does approach a context window, the fix is to know the window; there is no per-model context budget in the app today, and picking a new number here would only move the silence somewhere else.
Related issue
Refs #217 — the measured-leads document describes this class of defect (absence treated as a non-event); this is an instance of it found from a user report.
Type of change
Release impact
Desktop impact
Testing
New case in
agent-tools.test.ts: a 4000-word transcript (about half an hour of speech) must come back whole. It asserts the last word, not just the count — a cap keeps the head and drops the tail, so the tail is what proves it is gone. Restoring the.slice(0, 800)turns it red (expected […] to have a length of 4000 but got 800) and leaves the other sixty green.npm test: 137 files, 1626 passed.npm run wb:l0: unchanged (44 pre-existing failures, all the unversioned real fixture).npx tsc --noEmitand biome clean.The workbench already carried a
longTranscriptfixture whose comment documented this exact defect — never wired to a scenario. Its comment now states the behaviour instead of a to-do.Known follow-up
Even with the whole transcript,
replaceTimelineis forbidden for cut-silences by its own description, so the agent must emit oneaddTrimper silence, serially. On thirty minutes that is hundreds of round trips. That is the batch-tool lead in #217 and is not addressed here.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Documentation