Discovered vibevoice sitting at 3.2GB resident GPU memory with
loaded_models: [] and no idle-unload ever firing for it again. Root cause:
a load attempt had OOM'd partway through (competing with an unrelated
ollama process on the same GPU), so _MODEL_CACHE was never populated —
the idle-unload loop only clears that cache, so it had nothing to act on,
even though the partially-constructed model had already left memory
allocated. gc.collect()+empty_cache() don't reliably reclaim memory from
an interrupted from_pretrained() call.
Confirmed a plain process restart does fully reclaim it, so each worker
now treats any load failure as fatal: log it and os._exit(1), letting
supervisord's autorestart=true respawn a clean process immediately.
Verified with a bogus model id — worker exits, respawns, and passes the
smoke test right after.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every ASR/TTS worker (faster-whisper, qwen3, vibevoice, xtts) kept every
model it ever loaded resident in GPU memory forever, and switching to a
different model (e.g. a different whisper size) just added another one
alongside it rather than freeing the old one. Combined with the shared
24GB GPU, this made memory pressure only ever go up.
Now: only one model stays resident per worker at a time (loading a
different model_id evicts the previous one first), and the whole cache
(plus, for faster-whisper, the diarization pipeline) is dropped after 2
minutes of no requests. Verified end-to-end: the idle timer actually fires
and reclaims memory, and switching qwen3 models evicts the old one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors the ASR backend pattern exactly: a BACKENDS dict in
tts/router.py maps a backend name to its worker URL, so adding the
next backend is just a new worker file + venv + supervisord entry +
one dict line. XTTS-v2 runs in its own venv (--system-site-packages,
inherits base image torch/CUDA) as a new supervisord program on
port 8005.
XTTS is zero-shot voice cloning, so a reference-voice library was
added (/tts/voices CRUD, stored under /srv/tts/voices) — synthesis
requires picking a previously uploaded voice. Results and model
cache live under /srv/tts/{results,models-cache}, new quadlet
volumes, owned by the same 983:983 user as the existing /srv/asr
dirs.
Fixed two environment issues uncovered while getting XTTS to
actually run inside the container (non-root user, root-built venvs):
- coqui-tts only pins transformers>=4.57 with no ceiling, so pip
installed an incompatible 5.x; pinned to the last 4.x release.
- HOME defaults to /app (owned by root) for the container's runtime
user, so numba/matplotlib/torch cache writes failed; HOME is now
forced to /tmp in all three workers (faster_whisper, qwen3, xtts)
before any of those libraries get imported.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split the single ASR gateway into app/main.py (entrypoint) + app/core
(shared env helpers) + app/asr (all existing ASR logic, unchanged
behavior) + app/tts (placeholder router/config, no engine yet), so a
real TTS backend can be dropped in later without reshuffling ASR code.
API moved under /asr/* and /tts/* prefixes; UI gained a top-level
ASR/TTS tab switcher built on a reusable nested .tab-group mechanism,
with JS split into common.js/asr.js/tts.js.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
