Repository navigation
No ASR / transcription task; spoken-word defects are only inferable from tails and timing #188
Description
tool/endpoint: list_tasks — no transcription command among the task family repro: verify a spoken deliverable contains the words it should (e.g. a Bark or CSM TTS take). With no transcription task, the only checks are get_gallery_metadata envelope/duration and timing arithmetic. expected: a transcribe_audio/ASR task that returns the text of an audio asset (a small model task, consistent with how the engine already runs helper models). It would catch, directly, what currently must be inferred: dropped lines (Bark) and mid-sentence truncation (CSM) were both found only by measuring tails and doing words-per-second arithmetic. actual: no transcription command exists in list_tasks; nothing in the tool set returns what was spoken. impact: the class of defect where the words themselves are wrong is invisible to every current check — the strongest single capability gap for any speech deliverable. This is the highest-value audio task addition.
Activity
dkackman commentedon Sep 17, 2026
triage: work — not a duplicate; no transcription command exists (
dw/tasks/hasimage_to_textas the nearest helper-model task, and that is the pattern to copy: a transformers pipeline throughdw/tasks/model_cache.py,deviceconsumed viatask.device_for, registered indw/tasks/task.py, documented indocs/TASKS.md). Its own session, not batched with #185/#187: it adds a helper model (Whisper-class, via the transformers dependency the engine already has) that must be pulled onto lem and cached, and atext/plainresult the tester will read back withget_output_text. If the session judges the model choice / new weight download a step-7 decision, write the proposal instead of implementing; my read is that it is an ordinary task addition in theimage_to_textmould. Triaged by model opus via provider anthropic.feat(tasks): #188 - add transcribe_audio ASR task
fix(tasks): #188 - drop unused imports left in audio_utils.py
dkackman commentedon Sep 17, 2026
Implemented as an ordinary task addition per the triage comment:
transcribe_audio, a Whisper-class ASR task, so a TTS deliverable's word correctness (a dropped line, a mid-sentence truncation) can be checked directly against the text it was supposed to speak, instead of only inferred from duration/timing arithmetic.dw/tasks/audio_transcription.py(new) — dedicated module, followingtext_generation.py's pattern of a module-levelfrom transformers import pipeline as hf_pipelinerather than a lazy import inside the function. That choice wasn't just style-matching: a lazytransformers.pipelineimport inside a closure turned out not to be interceptable byunittest.mock.patch("transformers.pipeline")in this environment (transformers' lazy-module__getattr__machinery), which meant an early draft's tests silently downloaded a real Whisper model. Moving it to its own module with a module-level import fixed that.transcribe_audioindw/tasks/task.py,consumes_device=True.openai/whisper-base(ungated, already within thetransformersdependency).get_output_text.docs/TASKS.md, plus an example workflowworkflows/templates/transcribe-audio.json.tests/test_transcribe_audio.py(7 cases, all mocked — no model download).Commit:
3ae67b7ondevelop(merge commit3586fd3, feature commitea272ce).Deployed to
lemvia SSH:git pull origin develop, then MCP server restart viascreen -S 2147 -p 0 -X stuff $'\003'(three times) followed by relaunch. Confirmed healthy afterward:get_health→status: ok,hostname: lem,version: 0.4.0-beta.6;get_task("transcribe_audio")returns the expected schema. Restarted 2026-09-17T16:28Z.Proposed regression case (not written to any suite file — no harness repo checked out in this session): run
workflows/templates/transcribe-audio.jsonagainst a known TTS deliverable with a known transcript (e.g. an existing Bark/CSM output from a prior template run), assert the returned text is not empty and contains an expected keyword/phrase from the source prompt — a smoke-level check that the transcript reflects the deliverable's actual speech rather than a mocked/no-op response, which unit tests alone can't catch on a real deployment.model sonnet via provider anthropic
dkackman commentedon Sep 17, 2026
Verified on
lem, dw0.4.0-beta.6(get_health→status: ok,worker_alive: true), via the MCP interface only, in throwaway workspaceqa-verify-188. Tester: model opus via provider anthropic.What I ran
get_task("transcribe_audio")→ schema present (audiorequired;devicedefaultcpu;sample_rate;model_namedefaultopenai/whisper-base).list_workflows(shape="utility")liststemplates/transcribe-audio(kindstext, variablesinput_audio,model_name). Note it is bucketed underutility, nottext— fine, just so the next reader doesn't look for it undertextas I first did.templates/generate-speechwithtext="The purple elephant delivered seventeen umbrellas to the lighthouse on Tuesday."(bark-small, job9ac795d05425, 15 s) →templates/transcribe-audiowithinput_audio="output:templates/generate-speech/20260917-163903-da44b17d/GenerateSpeech-speak.0-0.0.wav"(jobf2ab2ceaff4b, 11 s).get_output_text→"The purple elephant delivered 17 umbrellas to the lighthouse on Tuesday."— every word of the script, Whisper's usual numeral normalisation aside.content_type: text/plain; charset=utf-8,truncated: false.device: cuda: inlineSpeakThenTranscribe:generate_speech(v2/en_speaker_3) →transcribe_audiowithaudio: "previous_result:speak",device: "cuda"(job052ef160620e, 17 s total). Output:"nine green bicycles waited outside the bakery until midnight."— exact.templates/transcribe-audiowithinput_audio="asset:qa-cast/ep6-cold-open.mp4"(an H3 clip from the shared library; jobba2d7e27d6ed, 2.6 s). Output:"Inventory says this hub is 90% full, so either the form is lying or you are lying."— coherent dialogue, so the video path in the schema doc works as stated.Result: matches
expected:— a transcription task exists, returns the spoken text astext/plainreadable viaget_output_text, and would catch a dropped line or truncation directly. Closing as verified. Regression case added asS-F042inregression-suite-smoke.md(steps 2 above, on theregression-smokeworkspace). Allqa-verify-188outputs deleted with the workspace.Develop (#241)