Every ASR/TTS worker (faster-whisper, qwen3, vibevoice, xtts) kept every
model it ever loaded resident in GPU memory forever, and switching to a
different model (e.g. a different whisper size) just added another one
alongside it rather than freeing the old one. Combined with the shared
24GB GPU, this made memory pressure only ever go up.
Now: only one model stays resident per worker at a time (loading a
different model_id evicts the previous one first), and the whole cache
(plus, for faster-whisper, the diarization pipeline) is dropped after 2
minutes of no requests. Verified end-to-end: the idle timer actually fires
and reclaims memory, and switching qwen3 models evicts the old one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Third ASR backend option alongside faster-whisper and qwen3. Builds
microsoft/VibeVoice from source (pinned to a specific commit, since it's
custom modeling code not in transformers' Auto* registry) into its own venv,
inheriting the base image's torch/CUDA. Comes with built-in speaker
diarization (VibeVoiceASRProcessor.post_process_transcription returns
per-segment speaker ids directly, no separate pyannote pass needed).
Verified end-to-end against a real recording: 200 OK, correct Korean
transcription, speaker labels populated.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
qwen3 backend: model list pointed at nonexistent HF repos (Qwen3-ASR-2B/8B
don't exist), and inference went through the generic HF ASR pipeline with
Whisper-only options (return_timestamps, task/language generate_kwargs) that
Qwen3-ASR's chat-style architecture doesn't support. Switched to the real
0.6B/1.7B-hf checkpoints and drive them via
processor.apply_transcription_request() + model.generate(). Also added an
ffmpeg pre-conversion step since the model's feature extractor can't decode
m4a via librosa.
faster-whisper backend: speaker diarization was broken by two pyannote.audio
API changes it hadn't caught up with (use_auth_token= renamed to token=, and
pipeline() now returns a DiarizeOutput wrapper instead of an Annotation
directly). Also pinned LD_LIBRARY_PATH for that worker so it picks up its own
venv's cuDNN instead of the older one shadowing it via the container's global
LD_LIBRARY_PATH, which crashed pyannote's GPU init.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The websocket handler now accumulates the full session's PCM audio
(not just the per-chunk buffer that gets discarded after each
worker call) and, on stop, writes it as a wav to UPLOAD_DIR plus a
result JSON to RESULT_DIR via the new _save_realtime_history()
helper — same shape /transcribe already writes, tagged
gateway_backend="realtime" so the history table visually
distinguishes it from file uploads. This reuses /asr/history and
/asr/uploads/{filename} as-is; no new endpoints needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors the ASR backend pattern exactly: a BACKENDS dict in
tts/router.py maps a backend name to its worker URL, so adding the
next backend is just a new worker file + venv + supervisord entry +
one dict line. XTTS-v2 runs in its own venv (--system-site-packages,
inherits base image torch/CUDA) as a new supervisord program on
port 8005.
XTTS is zero-shot voice cloning, so a reference-voice library was
added (/tts/voices CRUD, stored under /srv/tts/voices) — synthesis
requires picking a previously uploaded voice. Results and model
cache live under /srv/tts/{results,models-cache}, new quadlet
volumes, owned by the same 983:983 user as the existing /srv/asr
dirs.
Fixed two environment issues uncovered while getting XTTS to
actually run inside the container (non-root user, root-built venvs):
- coqui-tts only pins transformers>=4.57 with no ceiling, so pip
installed an incompatible 5.x; pinned to the last 4.x release.
- HOME defaults to /app (owned by root) for the container's runtime
user, so numba/matplotlib/torch cache writes failed; HOME is now
forced to /tmp in all three workers (faster_whisper, qwen3, xtts)
before any of those libraries get imported.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same Authlib-based pattern as v1: SessionMiddleware + core/auth.py
(login/callback/logout + require_login dependency), gating "/",
/asr/* and /tts/* behind Authentik (application slug "asr-v2",
provider pk 14). The websocket route can't use a FastAPI dependency
(no Request in its scope) so it's split into its own router and
checks websocket.session manually before accept().
v2 has no public domain yet, so the redirect_uri points at the
internal 172.30.1.41:18101 address over plain http — hence
https_only=False on the session cookie. Client id/secret and the
session secret live in /srv/asr/env/asr.env under ASR_V2_-prefixed
names (that env file is shared with v1, which already owns the
unprefixed OIDC_* names).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split the single ASR gateway into app/main.py (entrypoint) + app/core
(shared env helpers) + app/asr (all existing ASR logic, unchanged
behavior) + app/tts (placeholder router/config, no engine yet), so a
real TTS backend can be dropped in later without reshuffling ASR code.
API moved under /asr/* and /tts/* prefixes; UI gained a top-level
ASR/TTS tab switcher built on a reusable nested .tab-group mechanism,
with JS split into common.js/asr.js/tts.js.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
