pyannote.audio was still `>=3.1` — the same unbounded-constraint pattern
that silently broke qwen3 (transformers>=4.45.0 resolving to a version
without qwen3_asr support on a later rebuild). Confirmed it had already
drifted once this session (4.0.4 -> 4.0.7). Pinned to the exact version
the current diarization code (token=, .speaker_diarization) is verified
against.
asr.js's config-fetch failure path fabricated a hardcoded backend/model
list that could silently drift from the server's real one (this is
exactly how the wrong Qwen3-ASR-2B/8B model IDs stuck around). Replaced it
with a visible error instead of a second source of truth.
Added scripts/smoke_test.sh: hits every backend's /transcribe or
/synthesize directly with a synthetic clip. Every regression found this
session (wrong model IDs, pyannote API drift, cuDNN path conflict) would
have shown up here immediately instead of waiting for a user to hit it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Third ASR backend option alongside faster-whisper and qwen3. Builds
microsoft/VibeVoice from source (pinned to a specific commit, since it's
custom modeling code not in transformers' Auto* registry) into its own venv,
inheriting the base image's torch/CUDA. Comes with built-in speaker
diarization (VibeVoiceASRProcessor.post_process_transcription returns
per-segment speaker ids directly, no separate pyannote pass needed).
Verified end-to-end against a real recording: 200 OK, correct Korean
transcription, speaker labels populated.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
qwen3 backend: model list pointed at nonexistent HF repos (Qwen3-ASR-2B/8B
don't exist), and inference went through the generic HF ASR pipeline with
Whisper-only options (return_timestamps, task/language generate_kwargs) that
Qwen3-ASR's chat-style architecture doesn't support. Switched to the real
0.6B/1.7B-hf checkpoints and drive them via
processor.apply_transcription_request() + model.generate(). Also added an
ffmpeg pre-conversion step since the model's feature extractor can't decode
m4a via librosa.
faster-whisper backend: speaker diarization was broken by two pyannote.audio
API changes it hadn't caught up with (use_auth_token= renamed to token=, and
pipeline() now returns a DiarizeOutput wrapper instead of an Annotation
directly). Also pinned LD_LIBRARY_PATH for that worker so it picks up its own
venv's cuDNN instead of the older one shadowing it via the container's global
LD_LIBRARY_PATH, which crashed pyannote's GPU init.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The websocket handler now accumulates the full session's PCM audio
(not just the per-chunk buffer that gets discarded after each
worker call) and, on stop, writes it as a wav to UPLOAD_DIR plus a
result JSON to RESULT_DIR via the new _save_realtime_history()
helper — same shape /transcribe already writes, tagged
gateway_backend="realtime" so the history table visually
distinguishes it from file uploads. This reuses /asr/history and
/asr/uploads/{filename} as-is; no new endpoints needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split the single ASR gateway into app/main.py (entrypoint) + app/core
(shared env helpers) + app/asr (all existing ASR logic, unchanged
behavior) + app/tts (placeholder router/config, no engine yet), so a
real TTS backend can be dropped in later without reshuffling ASR code.
API moved under /asr/* and /tts/* prefixes; UI gained a top-level
ASR/TTS tab switcher built on a reusable nested .tab-group mechanism,
with JS split into common.js/asr.js/tts.js.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
