Discovered vibevoice sitting at 3.2GB resident GPU memory with
loaded_models: [] and no idle-unload ever firing for it again. Root cause:
a load attempt had OOM'd partway through (competing with an unrelated
ollama process on the same GPU), so _MODEL_CACHE was never populated —
the idle-unload loop only clears that cache, so it had nothing to act on,
even though the partially-constructed model had already left memory
allocated. gc.collect()+empty_cache() don't reliably reclaim memory from
an interrupted from_pretrained() call.
Confirmed a plain process restart does fully reclaim it, so each worker
now treats any load failure as fatal: log it and os._exit(1), letting
supervisord's autorestart=true respawn a clean process immediately.
Verified with a bogus model id — worker exits, respawns, and passes the
smoke test right after.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pyannote.audio was still `>=3.1` — the same unbounded-constraint pattern
that silently broke qwen3 (transformers>=4.45.0 resolving to a version
without qwen3_asr support on a later rebuild). Confirmed it had already
drifted once this session (4.0.4 -> 4.0.7). Pinned to the exact version
the current diarization code (token=, .speaker_diarization) is verified
against.
asr.js's config-fetch failure path fabricated a hardcoded backend/model
list that could silently drift from the server's real one (this is
exactly how the wrong Qwen3-ASR-2B/8B model IDs stuck around). Replaced it
with a visible error instead of a second source of truth.
Added scripts/smoke_test.sh: hits every backend's /transcribe or
/synthesize directly with a synthetic clip. Every regression found this
session (wrong model IDs, pyannote API drift, cuDNN path conflict) would
have shown up here immediately instead of waiting for a user to hit it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every ASR/TTS worker (faster-whisper, qwen3, vibevoice, xtts) kept every
model it ever loaded resident in GPU memory forever, and switching to a
different model (e.g. a different whisper size) just added another one
alongside it rather than freeing the old one. Combined with the shared
24GB GPU, this made memory pressure only ever go up.
Now: only one model stays resident per worker at a time (loading a
different model_id evicts the previous one first), and the whole cache
(plus, for faster-whisper, the diarization pipeline) is dropped after 2
minutes of no requests. Verified end-to-end: the idle timer actually fires
and reclaims memory, and switching qwen3 models evicts the old one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Third ASR backend option alongside faster-whisper and qwen3. Builds
microsoft/VibeVoice from source (pinned to a specific commit, since it's
custom modeling code not in transformers' Auto* registry) into its own venv,
inheriting the base image's torch/CUDA. Comes with built-in speaker
diarization (VibeVoiceASRProcessor.post_process_transcription returns
per-segment speaker ids directly, no separate pyannote pass needed).
Verified end-to-end against a real recording: 200 OK, correct Korean
transcription, speaker labels populated.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
qwen3 backend: model list pointed at nonexistent HF repos (Qwen3-ASR-2B/8B
don't exist), and inference went through the generic HF ASR pipeline with
Whisper-only options (return_timestamps, task/language generate_kwargs) that
Qwen3-ASR's chat-style architecture doesn't support. Switched to the real
0.6B/1.7B-hf checkpoints and drive them via
processor.apply_transcription_request() + model.generate(). Also added an
ffmpeg pre-conversion step since the model's feature extractor can't decode
m4a via librosa.
faster-whisper backend: speaker diarization was broken by two pyannote.audio
API changes it hadn't caught up with (use_auth_token= renamed to token=, and
pipeline() now returns a DiarizeOutput wrapper instead of an Annotation
directly). Also pinned LD_LIBRARY_PATH for that worker so it picks up its own
venv's cuDNN instead of the older one shadowing it via the container's global
LD_LIBRARY_PATH, which crashed pyannote's GPU init.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The websocket handler now accumulates the full session's PCM audio
(not just the per-chunk buffer that gets discarded after each
worker call) and, on stop, writes it as a wav to UPLOAD_DIR plus a
result JSON to RESULT_DIR via the new _save_realtime_history()
helper — same shape /transcribe already writes, tagged
gateway_backend="realtime" so the history table visually
distinguishes it from file uploads. This reuses /asr/history and
/asr/uploads/{filename} as-is; no new endpoints needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors the ASR backend pattern exactly: a BACKENDS dict in
tts/router.py maps a backend name to its worker URL, so adding the
next backend is just a new worker file + venv + supervisord entry +
one dict line. XTTS-v2 runs in its own venv (--system-site-packages,
inherits base image torch/CUDA) as a new supervisord program on
port 8005.
XTTS is zero-shot voice cloning, so a reference-voice library was
added (/tts/voices CRUD, stored under /srv/tts/voices) — synthesis
requires picking a previously uploaded voice. Results and model
cache live under /srv/tts/{results,models-cache}, new quadlet
volumes, owned by the same 983:983 user as the existing /srv/asr
dirs.
Fixed two environment issues uncovered while getting XTTS to
actually run inside the container (non-root user, root-built venvs):
- coqui-tts only pins transformers>=4.57 with no ceiling, so pip
installed an incompatible 5.x; pinned to the last 4.x release.
- HOME defaults to /app (owned by root) for the container's runtime
user, so numba/matplotlib/torch cache writes failed; HOME is now
forced to /tmp in all three workers (faster_whisper, qwen3, xtts)
before any of those libraries get imported.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
NPM proxy host + Let's Encrypt cert for speech.du5t.xyz are live, so
flip the session cookie to https_only=True (the redirect_uri and
Authentik provider config changes live outside this repo, in
/srv/asr/env/asr.env and the Authentik provider record).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The project now covers both ASR and TTS, so "asr-v2" no longer fits.
Renames: quadlet (asr-v2.container -> speech.container, container
name, image tag), build.sh, FastAPI/UI title, and the OIDC env var
prefix (ASR_V2_* -> SPEECH_*, matching the renamed Authentik
provider/application slug "speech"). Internal Python packages
(asr/, tts/, core/) were already named generically and don't change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same Authlib-based pattern as v1: SessionMiddleware + core/auth.py
(login/callback/logout + require_login dependency), gating "/",
/asr/* and /tts/* behind Authentik (application slug "asr-v2",
provider pk 14). The websocket route can't use a FastAPI dependency
(no Request in its scope) so it's split into its own router and
checks websocket.session manually before accept().
v2 has no public domain yet, so the redirect_uri points at the
internal 172.30.1.41:18101 address over plain http — hence
https_only=False on the session cookie. Client id/secret and the
session secret live in /srv/asr/env/asr.env under ASR_V2_-prefixed
names (that env file is shared with v1, which already owns the
unprefixed OIDC_* names).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split the single ASR gateway into app/main.py (entrypoint) + app/core
(shared env helpers) + app/asr (all existing ASR logic, unchanged
behavior) + app/tts (placeholder router/config, no engine yet), so a
real TTS backend can be dropped in later without reshuffling ASR code.
API moved under /asr/* and /tts/* prefixes; UI gained a top-level
ASR/TTS tab switcher built on a reusable nested .tab-group mechanism,
with JS split into common.js/asr.js/tts.js.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- faster-whisper, Qwen3-ASR, gateway 각 컴포넌트별 Python venv 분리
- 기본언어 한국어(ko)
- 처리내역 탭: 목록/상세/원본파일 재생/삭제
- 백엔드별 동적 모델 드랍다운
- /history, /uploads API 추가
- 기존 인스턴스(port 18100) 보존, 신규 port 18101
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
